I’m a literary snob. I tend to look down my nose at stories with a surprise ending, a “twist in the tail”. They seem to me gimmicky and superficial. And yet, I have been struck by a disturbing question: is surprise, after all, what writing is about?

A good story always has us eager to know what comes next. But now think about a bad story. It’s plodding and predictable. We know what’s going to come next—the bad king will be overthrown, the princess will marry the peasant, who will, of course, turn out to be a prince and the true heir. If we can predict what’s coming next, why would we read on? So, logically, a good story must have surprise.

This is what cognitive scientist Vera Tobin argues in her book Elements of Surprise. The book shows how the glitches in our cognition works informs the way successful plots are constructed. Jane Austen’s Emma, for example, turns on the consequences of the protagonist’s mistaken belief that she is an accomplished matchmaker.  She has great difficulty imagining that, once she has arrived at an interpretation of events, others could think otherwise. The surprise comes when she realises that she, after all, loves Mr Knightly.

Twists and jokes

Twists, or surprises, can work in different ways. In the classic version, the reader is led to believe things are one way, and then that belief is turned on its head. The scattering of clues must be sufficient for the ending to seem inevitable, and yet they must be sufficiently disguised not to be noticed. This, of course, is the recipe for the detective story. It is also the recipe for the joke.

Two gay men watch a beautiful woman hipping her way down the street. One sighs and turns to the other, saying “it’s at times like this you wish you were a lesbian”.

The mind is led down one path, only to be shown at the end a completely different meaning to the chain of events. The natural physiological response is the sudden explosive exhalation we call a laugh.

The problem with this version of the surprise is that it only works once. There is nothing to the lead-up that would hold our attention once we know how it will end. Once we know the punchline, once we know the butler did it, the magic is gone.

Twist in the tail and twist in the character

But I now realise there other kinds of surprise. And Tobin holds the key to them. She argues that two “flaws” in cognition are at the heart of stories, and at the heart of social interaction. The first is our difficulty in understanding that others may be different from us. The second is the “curse of knowledge” – we find it hard to model the mental states of people who don’t know something that we do. There is infinite room for fashioning dramatic misunderstandings in these two facts.

Since a character with a desire creates a plot, I suppose a character change is a form of “twist in the tail”. But it’s a more satisfying one, and one to which we will return again and again. Our reaction is less like that to a joke and more like that to a loved one.

But, I think there is more than just surprise available here. There is also delight and understanding. Fiction allows us to experience the one thing we can never really know—what the inside of someone else’s head feels like. When a character in whom we are invested undergoes change we experience a twist in the character. And, if the author has achieved the trick of making the change both inevitable and surprising, we understand something profound about ourselves and others.

It all makes sense

There is, perhaps, yet a third kind of story. It’s like a numinous form of the first, the twist story. In this kind, the writer reveals to us how the world works, or at least how their world works. There is an aesthetic pleasure of discovery when we understand how all the pieces fit together. Vicariously, it’s like experiencing the “aha moment” of a scientific discovery. When we have the whole picture, suddenly it all makes sense.

What kinds of surprise work best for you in a story?