We are so familiar with seeing, that it takes a leap of imagination to realize that there are problems to be solved. But consider it. We are given tiny distorted upside-down images in the eyes and we see separate solid objects in surrounding space. From the patterns of stimulation on the retina we perceive the world of objects and this is nothing short of a miracle.
—Richard L. Gregory, Eye and Brain, 1966
MOST OF OUR IMPRESSIONS of the world and our memories of it are based on sight. Yet the mechanisms that underlie vision are not at all obvious. How do we perceive form and movement? How do we distinguish colors? Identifying objects in complex visual environments is an extraordinary computational achievement that artificial vision systems have yet to duplicate. Vision is used not only for object recognition but also for guiding our movements, and these separate functions are mediated by at least two parallel and interacting pathways.
The existence of parallel pathways in the visual system raises one of the central questions of cognition, the binding problem: How are different types of information carried by discrete pathways brought together into a coherent visual image?
Visual Perception Is a Constructive Process
Vision is often incorrectly compared to the operation of a camera. A camera simply reproduces point-by-point the light intensities in one plane of the visual field. The visual system, in contrast, does something fundamentally different. It interprets the scene and parses it into distinct components, separating foreground from background. The visual system is less accurate than a camera at certain tasks, such as quantifying the absolute level of brightness or identifying spectral color. However, it excels at tasks such as recognizing a charging animal (or a speeding car) whether in bright sunlight or at dusk, in an open field or partly occluded by trees (or other cars). And it does so rapidly to let the viewer respond and, if necessary, escape.
A potentially unifying insight reconciling the visual system’s remarkable ability to grasp the bigger picture with its inaccuracy regarding details of the input is that vision is a biological process that has evolved in step with our ecological needs. This insight helps explain why the visual system is so efficient at extracting useful information such as the identities of objects independent of lighting conditions, while giving less importance to aspects like the exact nature of the ambient light. Moreover, vision does so using previously learned rules about the structure of the world. Some of these rules appeared to have become wired into our neural circuits over the course of evolution. Others are more plastic and help the brain guess at the scene presented to the eyes based on the individual’s past experience. This complex, purposeful processing happens at all levels of the visual system. It starts even ...