Band Limit and Appropriate Sampling in Microscopy
A. What Is "Sampling?"
In microscope images obtained with the help of electronic devices (e.g., CCD camera) or in a scanning microscope system, data are usually obtained (sampled) at equidistant coordinates in object space. The distance between these measurement positions is usually denoted as the "sampling distance." Current technology samples on orthogonal coordinates (rectilinear sampling) with appropriate sampling distances along the in-plane coordinates X, Y and the axial coordinate Z (for three-dimensional imaging).
For a deeper understanding of how to choose these distances correctly, it is useful to assume the intensity distribution near the detector (translated back to sample coordinates) to be a continuous function (Fig. 1). The equidistant sampling coordinates, at which the values of the image function are recorded, are depicted as dashed lines in Fig. 1. The fact that pixels have a discrete physical size is not considered here to keep the discussion simple. This pixel-size effect is described by a convolution and can be considered as a multiplicative modification of the optical transfer function (Sheppard et al., 1995). In the considerations given in this article, noise resulting from photons and/or the detector system is neglected.
B. Spatial Frequencies
Any physical function (e.g., as shown in Fig. 1) can be decomposed into a sum of sine waves, each with a different frequency, amplitude, and zero position (socalled phase). To exactly represent a function, this sum generally contains an infinite amount of components. Such decomposition is useful especially in optics, as images of (linear) optical systems have the interesting property of only transmitting sine waves of the sample up to a fixed limiting frequency. A periodic fluorescence distribution with a count of maxima per µm above this limit frequency will appear as a uniform constant in the image. Such systems are often called "band-limited" systems, as only a compact, limited band of frequencies (from zero to the maximal transmittable frequency) is transferred. How well a defined frequency is transferred is depicted in the "optical transfer function (OTF)" as outlined in Fig. 2 for inplane imaging in a fluorescence wide-field microscope. The "spatial frequency" given on the X axis of this graph denotes the number of maxima per meter for each sine wave. The sum of all sine waves yields the perfect image. The transfer strength of the OTF (Y axis) denotes how well a sample consisting of only one spatial frequency (one such sine wave) would be transferred to the detector.
C. Band Limit of Optical Systems
1. Fluorescence Microscopy
When considering the sampling properties of microscopic imaging, a physical model (Young, 1985) of the microscope system has to be constructed. Due to the laws of physics, this OTF is zero beyond a welldefined limiting frequency (see Sheppard et al., 1994). In the case of wide-field fluorescence microscopy, the limiting frequency is as follows:
with kxy,max denoting the maximal in-plane spatial frequency, λem the shortest detected vacuum wavelength of the fluorescence emission light, and NA = nsin(α) the numerical aperture defined by the refractive index n of the medium immersing the sample and the objective detecting light at an aperture half-angle of α.
Similarly, a limiting maximal transmittable axial spatial frequency kz,max can be given for sine waves with Z components as:
In confocal fluorescence microscopy, the illumination is confined to a diffraction-limited spot. At very small pinhole size, detection is also diffraction limited. A fluorophore needs to be excited and detected by the system, leading to a multiplicative process of excitation and detection. As a result, the aforementioned frequency limits have to be extended for the case of confocal fluorescence microscopy.
It should be noted that for larger pinhole sizes (>0.3 Airy discs, definition see Section III,B), the in-plane transfer function reaches values very close to zero above the spatial frequency stated by the wide-field limit. This explains why the in-plane resolution of confocal images achievable in practice is rarely substantially superior to wide-field images.
Because the images can be decomposed into the sine waves, it is useful to consider how an individual sine wave is sampled by the equidistant measurements. Figure 3 shows such a sine wave (solid black line) along with its values marked at equidistant sampling positions. Does such a measurement determine the frequency of the measured sine wave uniquely? As evident from Fig. 3 (dotted line), this is not the case, as multiple sine waves (these are called aliases) could possibly give the same measured values. However, if it is known that a maximal transmittable frequency exists, frequencies of sine waves in the image beyond that limit can be excluded as possible sources for this measurement. To guarantee a unique interpretation of a given measurement, all that needs to be done is to ensure all aliasing frequencies lie beyond the a priori known transmittable frequency limit.
In an acquired image, multiple frequencies are measured simultaneously, and the aforementioned should be true for all spatial frequencies possibly present in the image. The highest frequencies in the image (just below the frequency limit, e.g., kimg < kmax = 5µm-1) turn out to be most critical, as their lowest frequency alias (kalias = ks - kimg, e.g., at ks = 11µm-1, kalias is 6µm-1) appears at lower frequencies than aliases of other lower frequency image contributions. If the image is sampled at least at twice its highest contained spatial frequency (ks > 2kimg), aliasing can be avoided, as interpretation of the sampled values is unique because kalias is above the limiting frequency kma x. In other words, the sampling frequency (ks = 1/D), as defined by the sampling distance D, has to exceed the maximal transmittable frequency by a factor of two (ks > kNq = 2kmax). This limiting sampling frequency kNq is called the Nyquist frequency.
This means that all values of the continuous bandlimited function in between the sampled values can theoretically be predicted solely based on the knowledge of the band limit of the system and the sampled values (assuming an infinite amount of them and not considering noise). Sampling denser than this requirement stated earlier is termed oversampling; sampling sparser than the limit is termed undersampling.
Selection of the appropriate sampling distances is crucial for almost all optical systems with computerized image acquisition. In the field of optical microscopy, one has to discriminate among fluorescence-, transmission-, and reflection-based systems.
In the procedures suggested here, equations are stated for different microscopy arrangements.
A. CCD-Based Imaging Systems (in Plane)
Many imaging systems do not allow for any direct control over the size of the detector bins. CCD camerabased systems and digitizing micrographs on a scanner fall in this category. To select the appropriate sampling distance:
Let's say the pixel pitch of your camera is 7 µm and you are using no binning. The question is whether using a 100×, 1.3 NA oil (n = 1.516) immersion objective with the standard microscope tube lens (i.e., M = 100), the sampling distance in the focal plane corresponding to
B. Confocal Systems (in Plane)
Confocal microscopy usually allows for free control over the sampling distance in the object plane by selecting an appropriate magnification ("zoom") and image size in pixels. A notable exception to this is a Nipkov-type disc-scanning system employing a CCD camera. For such systems, the maximum sampling distances (dxy,max and dz,max) corresponding to a confocal system have to be selected, although the protocol for the CCD camera should be followed.
With the microscope parameters as stated in the wide-field example in Section III,A, confocal microscope illuminating at 488 nm
with a small detection pinhole setting, would allow a maximum in-plane sampling distance of
and the pixel-to-pixel spacing Dxy should be adjusted to a smaller value. For large pinholes, the following sampling distance should be acceptable:
C. Focus Series
For the acquisition of focus series (Z stacks), an additional sampling limit along the axial direction also needs to be obeyed by choosing an appropriate distance between neighboring image planes. To calculate the corresponding maximum plane-to-plane distance (dz,max), the aperture half-angle of the objective (αobj) has to be known. Because this value is often not stated, it has to be calculated from the numerical aperture and the refractive index (n) of the immersion medium:
Focus Series Examples
For the parameters of the wide-field fluorescence example just given, the appropriate spacing between successive image planes should be chosen to be below
with the aperture half angle being estimated from the NA (1.3) and the refraction index of oil (n - 1.516) :
Accordingly, the maximum spacing for confocal fluorescence microscopy (parameters as given in Section III,B) is calculated as
IV. PITFALLS: CONTRAST AND SAMPLING
Suppose the object and its image consist of a sine wave with a single spatial frequency (e.g., 200nm distance between two maxima). The sampling limit requires sampling at more than the double frequency (i.e., Dxy < 100nm). If this image is sampled too close to the limiting frequency, there may be a problem. Depending on the exact position of the object, one may be fortunate enough to sample a maximum, then a minimum, a maximum again, and so on or, if unlucky, always sample half the maximum (see example in Fig. 4). In the latter case, the result would be indistinguishable from an object with zero frequency. Even when sampling at distances slightly below the required minimum distance, a very low contrast can result for images of small size. Therefore, one should oversample, such that at least one full period of the resulting envelope amplitude modulation is captured. In other words, with M pixels along a spatial direction, the stated maximum sampling distance should be lowered by multiplication with the factor M/(M + 1). For common image sizes (e.g., 512 x 512), this factor is negligible, but it can be substantial when acquiring only few Z sections (in which case M is the number of sections).
A. When to Use Undersampling
For some applications the sampling limit does not need to be obeyed. If, for example, the task is to count homogeneously filled cells with a fluorescent dye using a 20x, 0.9 NA objective, it makes a lot of sense to seriously undersample the data. For a cell to be identified it might suffice to detect two adjacent bright pixels. Thus the required sampling can, in some cases, depend on the size of the object structure to image. Undersampling (e.g., by binning) can reduce the amount of acquired data (important in screening applications), reduce the readout noise, dark current, and sometimes enlarge the field of view. In the mentioned application the introduced aliasing effects should be tolerable.
In other applications (e.g., neuronal imaging of dendrites) it may be very tempting to undersample the data. However, thin structures (such as dendrites) may occasionally be lost in some pixels because they happen to lie between two sample points. Such missing gaps can then render a computer-based analysis difficult, as the structures appear ruptured and will also cause serious problems when successive computerized deconvolution is applied to data. In these cases it can even be better to reduce the numerical aperture of the objective than to seriously undersample at high NA.
B. When to Use Oversampling
For some gray value-based image processing tasks, oversampling is recommendable (Young, 1996, 1988; Verbeek, 1985; Verbeek and vanVliet, 1993); e.g., if the aim is the precise determination of object positions, one should oversample data by a factor of at least 1.5 (sample at a 1.5 times smaller pixel pitch as compared to the limits given earlier). Simulations revealed (Heintzmann, 1999) that determination of the center of mass can result in a significant systematical error even when sampled according to the stated sampling limits. Furthermore, a smaller sampling distance determines more precisely where each photon hits the detector, thus leading to a slightly more precise estimate of the particle position (Heintzmann, 1999). Oversampling can also be useful when successive deconvolution of data is planned (see below). Note that with oversampling, the photon dose delivered to the sample can still be kept constant. Acquired images may look inferior at a first glance, but they still contain all the necessary information. Such data always allow for successive binning, resampling, and/or smoothing to enhance the visual appearance.
C. Deconvolution: Out-of-Band Reconstruction Possibilities
When the computerized deconvolution is applied to data, it is often useful to oversample data during data acquisition. The reason is that constrained deconvolution is capable of "guessing" high-frequency components in the object structure that have not been acquired. This is enabled by the use of prior knowledge about the object, e.g., its positivity or smoothness (Sementilli et al., 1993). In principle, acquired data can be resampled, but this always involves loss of information about the photon statistics, or even interpolation errors can result. Although software might be able to reconstruct on a denser grid than raw data, there may be (depending on the algorithm) some resampling involved, which in turn can skew the statistics of the deconvolution procedure, thus leading to inferior results. As a rule of thumb, a two-fold oversampling (two times smaller sampling distances as compared to the given limits) should suffice even for advanced deconvolution software.
Appendix: Derivation of Cut-Off Frequencies
To obtain the equations given in the text, an expression for the cut-off spatial frequency was derived and the Nyquist theorem applied, calculating the maximum sampling distance as half the distance corresponding to this cut-off spatial frequency. Note that this derivation is valid for high NA vector theory.
Electric field components of a plane wave can be described by a single point in Fourier space. Its distance to the origin is proportional to the inverse wavelength. A lens forms its image by the constructive interference of converging parallel beams. It can thus be described by a "cap" (Gustafsson et al. 1995) in Fourier space (Fig. 5a).
For incoherent fluorescence wide-field imaging, the intensity in focus describing the point spread function (PSF) is obtained as the square of the absolute magnitude of the electric field distribution. Because the optical transfer function (OTF) is the Fourier transformation of the PSF, it can be obtained as an autocorrelation of Fig. 5a, which is identical to the convolution with itself mirrored at the origin in Fourier space. The resulting region of support is depicted in Fig. 5b. The corresponding distances are the reciprocal values, which have to be halved, yielding the respective maximum sampling distance in XY and Z directions.
In the case of confocal microscopy, photons have to be excited and detected. This leads to a multiplication of the probabilities of excitation PSF (corresponding to the OTF in Fig. 5b, but for λex) and the emission PSF (corresponding to the OTF in Fig. 5b). In Fourier space, this translates to a convolution of Fig. 5b as depicted in Fig. 6, yielding the appropriate equations for the confocal closed pinhole case. The size and shape of the pinhole is described by a multiplicative modification of the detection OTF with the Fourier transformed pinhole. The two-photon (no detector pinhole) derivation follows in a similar fashion, as its PSF is the square of the excitation PSF, which can be described by autocorrelation of Fig. 5b for λex.
When dealing with wide-field transmission, the situation is different. The image can no longer strictly be described as a convolution of the object with a point spread function. However, for incoherent imaging in transmission, such approximation still holds. Nevertheless it is preferable to think of this as a scattering problem. Neglecting multiple scattering, one can look for the scattering object vectors in Fourier space, which should be imaged by the system. Scattering theory states that the incoming wave vector plus the object vector yields the outgoing wave vector. Incoming and outgoing vectors are restricted by the Ewald sphere and the numerical aperture of the condensor and objective, respectively. As depicted in Fig. 7a the range of possible scattering vectors depends on both, the incoming light as defined by the numerical aperture of the condensor and the outgoing light as restricted by the objective.
The equation for confocal reflection microscopy is obtained by considering the possible scattering vectors when illuminating and detecting through the same objective (Fig. 7b).
B. Rieger, S. H6ppner, E. Lemke, I. T. Young, and T. M. Jovin are thanked for their help in revising this manuscript and C. J. R. Sheppard for fruitful discussions on sampling.
Gustafsson, M. G. L., Agard, D. A., and Sedat, J. W. (1995). Sevenfold improvement of axial resolution in 3D widefield microscopy using two objective lenses. Proc. SPIE 2412, 147-156.
Heintzmann, R. (1999). "Resolution Improvement of Biological Light Microscopic, data." Ph.D. thesis, Institute of Applied Physics, University of Heidelberg, Germany.
Sementilli, P. J., Hunt, B. R., and Nadar, M. S. (1993). Analysis of the limit to superresolution in incoherent imaging. J. Opt. Soc. Am. A 10, 2265-2276.
Sheppard, C. J. R. (1989). Axial resolution in confocal fluorescence microscopy. J. Microsc. 154, 237-241.
Sheppard, C. J. R., Gu, M., Kawata Y., and Kawata, S. (1994). Threedimensional transfer functions for high-aperture systems. J. Opt. Soc. Am. A 11, 593-598.
Sheppard, C. J. R., Gan, X., Gu, M., and Roy, M. (1995). Signal-tonoise in confocal microscopes. In "'Handbook of Biological Confocal Microscopy" (J. B. Pawley, ed.), pp. 363-371. Plenum Press, New York.
Verbeek, P. W. (1985). A class of sampling-error free measures in oversampled band-limited images. Pattern Recogn. Lett. 3, 287-292.
Verbeek, P. W., and van Vliet, L. J. (1993). Estimators of 2D edge length and position, 3D surface area and position in sampled grey-valued images. Biolmaging 1, 47-61.
Wilson, T. (1990). Confocal microscopy. In "Confocal Microscopy" (T. Wilson, ed.), pp. 1-64. Academic Press, London.
Young, I. T. (1985). Estimation of sampling errors. Cytometry 6, 273-274.
Young, I. T. (1988). Sampling density and quantitative microscopy. Anal. Quant. Cytol. Histol. 10, 269-275.
Young, I. T. (1996). Quantitative microscopy. IEEE Engineer. Med. Biol. Mag. 15, 59-66.
© 2018 Biocyclopedia | All rights reserved.