I think when you are building a good story, or any work of art that is experienced by an individual over time, there is a lot to be gained by working the same way. Anyone who has followed a television show for years only to be let down by the ending will agree with me. We owe it to our audiences to know where we are leading them, because every step of the story should be in service to that.
When you are reading a book (a good book) often it feels like being inside a corn maze. You are very alive within that moment, but you can't always see beyond the walls. As you proceed you gain more information, but are still hemmed in to what the writer chooses to reveal. It isn't until the end that you realize how each of those turns brought you to this moment. If it is a stupid maze then you feel let down, quickly get into your car and drive off. If it is a good maze, then there is a moment where you stand there completely aghast at how all those little pieces and parts inexplicably brought you to this perfect moment.
There have been many times that I've read a book, realizing as I read it that it is a pretty good book. Well-written, good characters, lots of beautiful little moments. Happily I continue along until I get to the ending, where the writer just completely eviscerates me with one amazing paragraph that puts everything prior into a completely new light, bowling me over with the power of what I just read. These are the best books. These are my favorite books. These are the books that sit in my heart and haunt me. (Selfishly, these are also the books that I want to write.)
Connie Willis does this and it is one of the reasons I love her writing so much. She doesn't seem to get talked about these days outside the museum of science fiction, but I suggest anyone who hasn't read her work pick up Passage, Lincoln's Dreams, or Doomsday Book. The writer knows what she is doing, even when the reader is completely lost.
Henry James does the same thing, but in a different way. What I love about his stories is that sometimes you don't know the characters until the very last paragraph. You think you do, but in one beautiful moment they turn around and look you square in the face, saying, "No, you don't know the first thing about me. This is who I am and it is far grander, stranger, and more pathetic than you could have guessed." Suddenly their world is larger than yours.
Why do I so love my books this way? Maybe because this is my hope for life as well. So often things seem random, meaningless, messy, dull, and mean. If I can cling to the hope that they aren't meaningless at all, but leading to a magnificent final paragraph, then it makes it easier to accept the journey.
To some extent, isn't that what we are trying to do when we make art? Put the world back into order and prove that it wasn't a lame maze, but a totally awesome one?
I Guess That's Why They Call It The Blues - Elton John