Conventions

The idea of conventions, once separated out from preconceptions, is beginning to make sense to me. Whilst conventions and preconceptions spend much time in bed with one another, both are worth studying independently; they exist in reader-land as expectations - as goalposts - and only a foolish writer would renounce their existence.

What I would like to consider here is the idea of the genre convention:
'To anticipate the anticipations of the audience you must master your genre and its conventions.' [Source: Robert McKee's Story.]
'A convention is an understanding between writer and reader about certain details of the story.'
[Source: Crawford Kilian.]

As I begin work on novel #4, I am considering genres and what the reader expects from them. I can see that such expectation dissolves the value of many critiques/assessments: A devout reader of fantasy might take his expectations across to a thriller, therefore finding himself in confusion. This isn't progressing as I would expect!

An author who reads and subsequently writes in a narrow genre takes with him a subconscious understanding of the genre's conventions. We are familiar with the fledgling writer who flouts his studies and preparations, preferring to simply sit and write, creating a work that holds together because that author has subconsciously transferred his own expectations across to his writing.
Yes, it happens occasionally and we end up with the celebrated teenage author or the single mother who strikes gold with her first attempt.
Moreover, a per-genre convention guide is a pretty elusive beast. What are the expectations of any given genre?
We are encouraged to read everything and anything, and this is perhaps the best we can do: in this way we can gain valuable perspectives beyond those with which we may have become comfortable; we can compare and contrast conventions; we can identify key conventional motifs.
I remember my first few forays in lit fic. Certainly Hemingway and Ali Smith scared the wits from me. But I quickly acclimatized: I came to expect particular elements, although I would be hard-pressed to outline those elements from the top of my head.
Wikipedia has some ideas.
McKee looks at audience positioning, creative limitations, analyzes a few movies, and ends the chapter on endurance.
Crawford Kilian has a stab at outlining the conventions of the thriller.

With all of this food for thought, I can only conclude that the author must write in a genre with which he is familiar - in which the expectations have become second-nature.
The idea of trusting instinct always frightens me. This morning I was reading about one pro author's first encounter with distrust of instincts. Reworking a manuscript that had once gone through a nightmarish editorial process, the author is now filled with doubt.
I would like to believe that study and practice and a modicum of talent are enough to carry one through. But the deeper my journey takes me, the more I begin to doubt this ...

At the heart of this post is a fear: How can I understand my readers? What if I do not understand them? What if I am unable to connect with them?