It may turn out that the instant you are now experiencing – really, any instant you experience – is an amalgamation of the previous 15 seconds of consciousness. This is the conclusion of a paper* published in the recent journal Nature Neuroscience by scientists from MIT and UC-Berkeley. They presented a series of test images to subjects and recorded their responses to questions asked about them. The experiment plays off two areas of higher brain functions: visual input and visual processing.

Most people tend to assume that visual input is nothing more than our eyes acting as motion picture cameras that feed a constant visual stream to the visual cortex. In fact, our eyes are highly developed image processing engines. I learned this a little over a decade ago when I was involved in research that modeled the complex activity of the visual cortex of fresh water turtles when simple patterns were shown to them. The experimental results of the research, as well as many similar experiments done with primate and human subjects, reveal that the eye acts nothing like a motion camera. At the back of the eyeball is the retina. My grade-school level science had me thinking that the retina was little more than a screen on which our eye projected the view of the outside world. The retina is actually covered by thousands of optical nerves of various types. Most of us know about the two main types: The rods that respond to levels of lightness and darkness and give us the dynamic range to see in all kinds of lighting conditions; and the cones that respond to the three primary colors and give us a sense of color vision.

But there are many different types of rods and cones. For example, there are optic nerves that will only excite (send a signal down the optic nerve through the back of the eye and into the brain) if the visual field contains straight vertical lines, say a tall skyscraper or a tree. There are others that only excite if there are horizontal lines and still others for straight lines of various orientations. Then there are optic nerves that only excite if there are circular shapes in the field of view. Some will only excite if something moves from left to right in front of the viewer. The list goes on and on but hopefully you get the main idea: The eye does not present a snapshot, frame-by-frame view of the external world to the brain. It breaks each scene up into component colors, shapes and motion types and only sends this subset data to the brain. The amount of data to be processed by doing this is considerably less. Those of you who are familiar with modern image encoding methods (mpeg for example) are already familiar with this concept. The movies we watch on our DVDs are not frame-by-frame encodings of actual films. That would require more data than can actually be stored on modern DVDs. Instead, movie frames are broken down to their basic shape and color data and this subset of data is recorded.  Moreover imagine that you are staring at the same scene for an extended period of time, perhaps you are enjoying your favorite O’Keeffe painting at the local museum. In this case most of your eyes tend to do nothing at all! To save itself a lot of unnecessary work our visual system decides that if nothing is changing in a scene then it will stop trying to process it and the optical nerves will stop firing and sending data back. In reality we cannot do this for long periods of time because even when we admire a work of art, our eyes go through saccadic motion – a rapid scanning of the visual field that our eyes have evolved to undertake every few microseconds to look for interesting or threatening objects in our field of view. Thus even when we stare at the clouds in the sky the motion of our eyes will update the view every so often as to keep the image data flowing but it is far less than sending a constant stream of data to the visual cortex inside the brain.

Now that you have some idea of how our eyes process visual data let’s move onto the more complex bit of neuroscience that deals with how we perceive this data. The visual cortex is where this occurs. It takes in the data coming across the optic nerves and puts the various fragments together to reform the whole, but at each moment of time only a partial amount of data regarding the visual scene is being assembled. If we were able to somehow pause this playback at a very precise time and look at the reconstructed scene we would only see a little bit of the image and a lot of blank areas left to be filled in. Resuming play, some of those missing areas would then get filled in but another “pause” and we see that some of the areas that were previously filled in are now blank.  In other words, for very small precise moments of time what we perceive as a complete field of view is really only partially filled in live data. That we perceive an entire whole scene is a result of the neurological tango between the visual cortex and our memory system. Visual memory is temporary enough to keep some of the scene around while the rest gets filled in so that we are lead to believe we are seeing everything all at once.

And just how much time is required for us to experience an instant of visual acuity? That is what these researchers are attempting to find out.  They set up a simple experiment in which a series of straight line segments are flashed on a screen and then disappear. Every image shows a different series of straight lines rotated with respect to the previous image. (These are known as Gabor patches.) A new image appears every 5 seconds. After each image appears and disappears the subject is asked about the orientation of the lines they just saw. They get the answers mostly correct. They get the general orientation correct, meaning that the most recent image is still fresh in their memory but the orientation is always a bit off. In fact, it is nearly always skewed towards the lines shown in the previous two images, meaning that those other images are somehow still persistent in the subjects visual image and that they continue to play a role in how the subject interprets the current scene. Images flashed more than 15 seconds previously have not been found to have any effect.

Thus what the researches have found is evidence, as far as our visual system is concerned, that “instants” of time are really a summation of the previous 15 seconds of our experience.

*Serial dependence in visual perception, Jason Fischer and David Whitney, Nature Neuroscience (2014), published 30 March 2014.