Visual resolution appears to be limited by neural sampling everywhere in the visual field except the very center, where optical limitations dominate. Consequently, visual acuity represents a non-invasive estimate of the spatial density of the mosaic of photoreceptors (in parafoveal retina) or ganglion cells (in peripheral retina) of the living human eye. Recent experiments have paved the way for clinical applications of this approach to the problem of monitoring the spatial densities of neurons in a variety of retinal diseases.
Helmholtz laid the foundation for the sampling theory of visual resolution over a century ago when he argued that resolution requires at least one relatively unstimulated neuron between two relatively stimulated neurons 1. Although Helmholtz formulated his rule in the context of resolving two points of light, it applies equally well to the case of resolving the bars of a sinusoidal grating, illustrated in Fig. 1. Figure 1A depicts a sinusoidal grating being sampled by three rows of visual neurons. The spatial pattern of neural responses constitutes a "neural image" that represents the stimulus within the visual system. The neurons in the top row are packed tightly enough to exceed the requirements of Helmholtz's rule, the minimum sampling density required is shown in the middle row, but the bottom row of neurons are too widely spaced to satisfy Helmholtz's rule and consequently the pattern is undersampled.
Figure 1. (A) Three sampling regimes. Top row, oversampled; middle row, critically sampled; bottom row, undersampled (B) Neural undersampling of a two-dimensional grating misrepresents orientation (dashed lines) and spatial frequency.
In one respect the neural image and the optical image it represents are fundamentally different: the optical image formed on the retina is continuous whereas the neural image is discrete. Therefore, to appreciate the spatial content of the neural images depicted in Fig. 1A, it is helpful for the reader to mentally interpolate between sample points to form an envelope of modulation. Clearly this envelope accurately represents the spatial frequency of the stimulus for the upper and middle neural images illustrated. In fact, given the proper method of interpolation 2, the envelope will exactly reconstruct the original signal provided that the sampling process is error-free and that the sampling density satisfies Helmholtz's Rule. On the other hand, if Helmholtz's Rule is violated then the envelope of modulation in the neural image will yield a false representation of the stimulus waveform, as occurs in the bottom neural image of Fig. 1A. This unavoidable misrepresentation of the stimulus due to undersampling is called aliasing.
In short, according to the sampling theory of visual resolution, the highest spatial frequency which a neural array can faithfully represent (the Nyquist frequency) is one-half the neural sampling frequency. Stimuli beyond the Nyquist frequency may still modulate the neural image, but the resulting pattern of neural activity will misrepresent the physical stimulus as a lower-frequency alias. Thus, the sampling limit to visual resolution is characterized by the onset of aliasing. In the case of undersampling by two-dimensional sampling arrays, the neural image may also misrepresent the orientation of the stimulus pattern, as shown in Fig. 1B (where the strength of neural response is coded by the luminance of the circles at each sample point). For the example illustrated, the neural image indicates the presence of an obliquely oriented pattern with a lower spatial frequency than is actually present on the retina.
Figure 2. Filtering model of the limit to visual resolution. The spatial frequency spectrum of the object (left) is sampled by an array of overlapping receptive fields (middle) resulting in the attenuation of contrast sensitivity of the observer (right) for frequencies beyond the Nyquist limit fN of the sampling array. Dashed curve at right excludes the effect of filtering, solid curve includes filtering.
Although Helmholtz's Rule is well known to every student of visual science, until relatively recently there was little evidence that the rule actually applies to human vision. That is, there was scant evidence that the tell-tale signs of aliasing accompanied the end of visual resolution as we know it. In the absence of compelling evidence of neural undersampling, numerous competing theories rose to prominence which implied that the sampling limit is never attained in vision because spatial filtering attenuates high frequencies, thus preventing aliasing. According to this filtering model of the limit to visual resolution, contrast sensitivity is less than the absolute threshold of unity for spatial frequencies above the Nyquist limit. One physiological arrangement consistent with the filtering model is illustrated in Fig. 2. Individual receptive fields of sampling units in this model are relatively large when plotted in object space, perhaps due to large anatomical dendritic fields or to blurring by the eye's optical system. Consequently, receptive fields overlap extensively to provide redundant coverage of the visual field. Spatial summation over such receptive fields will reduce the cutoff spatial frequency of individual cells to zero before the Nyquist limit of the array is exceeded 3. Although aliasing is avoided in a filtering-limited visual system, the penalty is a loss of contrast sensitivity at frequencies below the Nyquist limit 4 (right-hand panel of Fig. 2). Conversely, for neural undersampling to be the limiting factor for visual resolution, contrast sensitivity must be greater than unity over a range of spatial frequencies beyond the Nyquist limit. This would be achieved by an array of sampling elements with relatively small receptive fields with a low coverage factor, as illustrated in Fig. 3.
Qualitative evidence that peripheral resolution is sampling limited is provided by reports of both spatial and motion aliasing reported independently by several groups using a variety of test stimuli and viewing conditions (see 5 for refs.). Quantitative support for the sampling hypothesis includes evidence that detection acuity can exceed resolution acuity by an order of magnitude in peripheral vision and that contrast sensitivity is much greater than unity at the resolution limit 5. Saturation of resolution acuity as contrast increases is additional evidence that peripheral resolution is limited by undersampling 5. Finally, the close correlation between psychophysical and anatomical estimates of sampling density of the cone mosaic in the parafoveal retina 6 and the ganglion cell mosaic in the periphery 7 suggests that not only is visual resolution sampling-limited, the limiting array is in the retina itself.
Figure 3. Sampling model of the limit to visual resolution. Similar to Fig. 2 except sampling array has low coverage factor. Energy in the aliasing zone of the object frequency spectrum is undersampled and therefore misrepresented as low spatial frequencies which are necessarily below the Nyquist frequency. Dashed curve at right excludes the effect of sampling; symmetric solid curve includes sampling.
The rationale for clinical applications of the sampling theory of resolution is that acuity, when measured with a sampling-limited task, is an estimate of the density of retinal neurons. Clinical exploitation of this approach for the detection and monitoring of photoreceptor loss in central retina, or ganglion cell loss in peripheral retina, will require additional research to determine the sensitivity of the technique to various clinical conditions. Experience with normal observers gives us reason to be optimistic since even subtle features of retinal anatomy such as naso-temporal asymmetry and the visual streak, a region of slightly elevated density of retinal ganglion cells near the horizontal meridian, are revealed by peripheral acuity measurements 8. Another concern is whether the measurement of resolution acuity using a sampling-limited technique is reliable enough for clinical use. For trained subjects, changes in acuity as small as 10% are measurable 5 but the reliability of measurements obtained from untrained patients remains to be discovered. Whether it is necessary to correct the sizable refractive errors of the peripheral field in order to achieve sampling-limited performance must also be determined. Recent experiments have shown that although detection acuity varies strongly with defocus, resolution acuity for high contrast targets is robust against refractive blur, being independent of defocus over a range of several diopters 9. This suggests that careful correction of peripheral refractive error may not be necessary in practice, provided targets are of high contrast and approximately in focus. Fortunately there is a built in check to see if close enough is good enough: so long as detection acuity exceeds resolution acuity, aliasing is indicated and we infer that resolution remains sampling limited. Optimal target configuration and psychophysical methods for clinical work also need to be determined by future research.
In short, much work remains to be done to develop these emerging ideas from laboratory science into useful tools for patient care. There could be no more auspicious time to begin that work than now, at this celebration of the career of Jay Enoch.