While I agree with Hesse, Dostoyevsky, Garcia Marquez & Harper Lee as though provoking novelists, the most for me was The World in the Evening by Christopher Isherwood. I read it while travelling in California and identified with the main character so strongly at the time that it hurt. I remember being in Yosemite feeling completely overwhelmed and unable to tell the Witness friends I was with because it meant telling them the deep dark secret about my sexuality. Took me 10 more years to tell them.
Have to add Sunset Song by Lewis Grassic Gibbon but unless you're Scottish it's unlikely you'll have heard of it.