"But even then, the author can throw something into the story that will just kick me out of enjoying it"
Totally with you there. For me, it's a suspension-of-disbelief problem. I grew up with the belief that even fantasy should be internally consistent. If the author offers me an implausible scenario, I need something that will assure me that, within this world she's creating, the usual rules of human nature have been suspended for good reason.