Band Limit and Appropriate
Sampling in Microscopy I. INTRODUCTIONA. What Is "Sampling?"In microscope images obtained with the help of electronic devices (e.g., CCD camera) or in a scanning microscope system, data are usually obtained (sampled) at equidistant coordinates in object space. The distance between these measurement positions is usually denoted as the "sampling distance." Current technology samples on orthogonal coordinates (rectilinear sampling) with appropriate sampling distances along the in-plane coordinates X, Y and the axial coordinate Z (for three-dimensional imaging). For a deeper understanding of how to choose these distances correctly, it is useful to assume the intensity distribution near the detector (translated back to sample coordinates) to be a continuous function (Fig. 1). The equidistant sampling coordinates, at which the values of the image function are recorded, are depicted as dashed lines in Fig. 1. The fact that pixels have a discrete physical size is not considered here to keep the discussion simple. This pixel-size effect is described by a convolution and can be considered as a multiplicative modification of the optical transfer function (Sheppard et al., 1995). In the considerations
given in this article, noise resulting from
photons and/or the detector system is neglected.
B. Spatial FrequenciesAny physical function (e.g., as shown in Fig. 1) can be decomposed into a sum of sine waves, each with a different frequency, amplitude, and zero position (socalled phase). To exactly represent a function, this sum generally contains an infinite amount of components. Such decomposition is useful especially in optics, as images of (linear) optical systems have the interesting property of only transmitting sine waves of the sample up to a fixed limiting frequency. A periodic fluorescence distribution with a count of maxima per µm above this limit frequency will appear as a uniform constant in the image. Such systems are often called "band-limited" systems, as only a compact, limited band of frequencies (from zero to the maximal transmittable frequency) is transferred. How well a defined frequency is transferred is depicted in the "optical transfer function (OTF)" as outlined in Fig. 2 for inplane imaging in a fluorescence wide-field microscope. The "spatial frequency" given on the X axis of this graph denotes the number of maxima per meter for each sine wave. The sum of all sine waves yields the perfect image. The transfer strength of the OTF (Y axis) denotes how well a sample consisting of only one spatial frequency (one such sine wave) would be transferred to the detector.
C. Band Limit of Optical Systems1. Fluorescence MicroscopyWhen considering the sampling properties of microscopic imaging, a physical model (Young, 1985) of the microscope system has to be constructed. Due to the laws of physics, this OTF is zero beyond a welldefined limiting frequency (see Sheppard et al., 1994). In the case of wide-field fluorescence
microscopy, the limiting frequency is as follows:
with k denoting the maximal in-plane spatial frequency, λ_{xy,max}_{em} the shortest detected vacuum wavelength of the fluorescence emission light, and NA = nsin(α)
the numerical aperture defined by the refractive index
n of the medium immersing the sample and the objective
detecting light at an aperture half-angle of α. Similarly, a limiting maximal transmittable axial spatial frequency k can be given for sine waves
with _{z,max}Z components as:
In confocal fluorescence microscopy, the illumination is confined to a diffraction-limited spot. At very small pinhole size, detection is also diffraction limited. A fluorophore needs to be excited and detected by the system, leading to a multiplicative process of excitation and detection. As a result, the aforementioned frequency limits have to be extended for the case of confocal fluorescence microscopy. It should be noted that for larger pinhole sizes (>0.3 Airy discs, definition see Section III,B), the in-plane transfer function reaches values very close to zero above the spatial frequency stated by the wide-field limit. This explains why the in-plane resolution of confocal images achievable in practice is rarely substantially superior to wide-field images. 2. AliasingBecause the images can be decomposed into the sine waves, it is useful to consider how an individual sine wave is sampled by the equidistant measurements. Figure 3 shows such a sine wave (solid black line) along with its values marked at equidistant sampling positions. Does such a measurement determine the frequency of the measured sine wave uniquely? As evident from Fig. 3 (dotted line), this is not the case, as multiple sine waves (these are called aliases) could possibly give the same measured values. However, if it is known that a maximal transmittable frequency exists, frequencies of sine waves in the image beyond that limit can be excluded as possible sources for this measurement. To guarantee a unique interpretation of a given measurement, all that needs to be done is to ensure all aliasing frequencies lie beyond the a priori known transmittable frequency limit.
In an acquired image, multiple frequencies are measured simultaneously, and the aforementioned should be true for all spatial frequencies possibly present in the image. The highest frequencies in the image (just below the frequency limit, e.g., k < _{img}k =
5µm_{max}^{-1}) turn out to be most critical, as their lowest frequency
alias (k = _{alias}k - _{s}k, e.g., at _{img}k = 11µm_{s}^{-1}, k is
6µm_{alias}^{-1}) appears at lower frequencies than aliases of
other lower frequency image contributions. If the
image is sampled at least at twice its highest contained
spatial frequency (k > 2_{s}k), aliasing can be avoided,
as interpretation of the sampled values is unique
because kalias is above the limiting frequency kma x. In
other words, the sampling frequency (_{img}k = _{s}1/D), as
defined by the sampling distance D, has to exceed the
maximal transmittable frequency by a factor of two
(k > _{s}k_{Nq} = 2k). This limiting sampling frequency
kNq is called the Nyquist frequency._{max}This means that all values of the continuous bandlimited function in between the sampled values can theoretically be predicted solely based on the knowledge of the band limit of the system and the sampled values (assuming an infinite amount of them and not considering noise). Sampling denser than this requirement stated earlier is termed oversampling; sampling sparser than the limit is termed undersampling. II. INSTRUMENTATIONSelection of the appropriate sampling distances is crucial for almost all optical systems with computerized image acquisition. In the field of optical microscopy, one has to discriminate among fluorescence-,
transmission-, and reflection-based systems.III. PROCEDURESIn the procedures suggested here, equations are stated for different microscopy arrangements. A. CCD-Based Imaging Systems (in Plane)Many imaging systems do not allow for any direct control over the size of the detector bins. CCD camerabased systems and digitizing micrographs on a scanner fall in this category. To select the appropriate sampling distance: - Obtain the pixel pitch (P) of your CCD camera
from the manufacturer/supplier. The pixel pitch is the
distance in
*X*and*Y*directions between successive pixels. Usually this is in the range of 5 to 20 µm. When binning is used during image acquisition, the pixel pitch has to be multiplied with the binning factor (e.g., with 2 × 2 binning: multiply the pixel pitch by 2). When digitizing micrographs, a corresponding value is given by either the pixel-to-pixel distance after scanning or the resolution of the micrograph, whichever value is bigger. When a micrograph is scanned, care has to be taken to account for all magnification factors to finally obtain a pixel size corresponding to the image plane. - Make a list of available objectives. Note magnification
(e.g., 40 or 63×) and numerical aperture (e.g., 0.9
or 1.3) for each of them. These values are usually
engraved on the side of the objectives. If any postmagnification
system (e.g., Zeiss Optovar) is available
on your microscope, multiply the objective magnification
with the appropriate factor. The obtained total
magnification is called
*M*. - Calculate the in-plane sampling distance (
*D*) as dictated by the detector pixel pitch in the object plane for each objective, dividing the pixel pitch by the magnification:_{xy}
**D**_{xy}=**P****M**
- Calculate the maximum sampling distance (dmax)
in the object plane from the numerical aperture of the
objective and the vacuum wavelength of light used for
imaging:
**Wide - field fluorescence :***d*=_{xy,max }**λ**_{em}**4****NA**_{obj}
**Transmission or phase microscopy:***d*=_{xy,max }**λ**+**2(NA**_{obj }**NA**_{cond}**)** In wide-field illumination microscopy systems, λ_{em}should be the shortest detected emission wavelength (e.g., FITC λ_{em}= 500nm); in transmission microscopy, the shortest transmitted wavelength should be used (e.g., blue at λ = 400nm). Note that the numerical aperture of the condensor (*NA*) contributes equally as the objective numerical aperture to the final resolution, although the contrast may suffer in transmission microscopy with a high condensor numerical aperture. Also note that the value_{cond}*NA*is taken for a fully open condensor aperture; a closed condensor aperture enhances the contrast but reduces_{cond}*NA*and thus the resolution._{cond} - Ensure that
*D*<_{xy}*d*by selecting the appropriate objective and/or postmagnification optics from the list._{xy,max}
CCD ExampleLet's say the pixel pitch of your camera is 7 µm and you are using no binning. The question is whether using a 100×, 1.3 NA oil ( n = 1.516) immersion objective
with the standard microscope tube lens (i.e., M = 100),
the sampling distance in the focal plane corresponding to
B. Confocal Systems (in Plane)Confocal microscopy usually allows for free control over the sampling distance in the object plane by selecting an appropriate magnification ("zoom") and image size in pixels. A notable exception to this is a Nipkov-type disc-scanning system employing a CCD camera. For such systems, the maximum sampling distances ( d and _{xy,max}d) corresponding to a confocal
system have to be selected, although the protocol for
the CCD camera should be followed.
_{z,max}- Calculate the maximum sampling distance
(Sheppard, 1989; Wilson, 1990; Sheppard
*et al.*, 1994) from the parameters of the objective (see CCD procedure for definitions):
**Confocal fluorescence (small pinhole):**=*d*_{xy,max}**λ**_{eff}**, λ**=_{eff}**1****4****NA**_{obj}**1****+****1****λ**_{ex}**λ**_{em}
**Confocal fluorescence (large pinhole):**=*d*_{xy,max}**λ**_{eff}**4****NA**_{obj}
**Confocal two-photon fluorescence (no pinhole):**=*d*_{xy,max}**λ**_{ex}**,****8****NA**_{obj} with λbeing the irradiating wavelength (usually in the infrared)._{ex}
**Confocal reflection:**=*d*_{xy,max}**λ****4****NA**_{obj} The strict theoretical limit even for large pinholes is the value given for small pinhole size. However, because the lateral high-frequency content for larger pinholes is negligible and usually lies well below the noise level, the "wide-field fluorescence" in-plane equation (as given earlier) can be applied, replacing the emission wavelength with the excitation wavelength. As a rule of thumb, consider pinhole sizes below 0.5 Airy discs in the pinhole plane as being small and sizes above 1.5 Airy discs as being large. In object space coordinates one Airy disc diameter (the first dark ring of a diffraction limited spot, assuming low NA) is
**1.22****λ**_{em}**.****NA**_{obj} To compare with actual pinhole sizes, the pinhole has to be translated to the object space coordinates by the appropriate demagnification factor, if the pinhole size is not stated in Airy disc units in the microscope operating software. - For beam-scanning or object-scanning confocal
systems, the in-plane sampling distance (
*D*) is usually stated somewhere on the screen. It can also be calculated from the image size in the object plane (_{xy}*S*_{img}) and the number of pixels (*N*_{pix}) along*X*or*Y*:
=**D**_{xy}*S*_{img}**.***N*_{pix}-1 This distance should be selected to be below the dxy, max calculated earlier.
Confocal ExampleWith the microscope parameters as stated in the wide-field example in Section III,A, confocal microscope illuminating at 488 nm with a small detection pinhole setting, would allow a maximum in-plane sampling distance of
and the pixel-to-pixel spacing D should be adjusted
to a smaller value. For large pinholes, the following
sampling distance should be acceptable:_{xy}
C. Focus SeriesFor the acquisition of focus series ( Z stacks), an
additional sampling limit along the axial direction also
needs to be obeyed by choosing an appropriate distance
between neighboring image planes. To calculate
the corresponding maximum plane-to-plane distance
(d), the aperture half-angle of the objective (α_{z,max}) has
to be known. Because this value is often not stated, it
has to be calculated from the numerical aperture and
the refractive index (_{obj}n) of the immersion medium:
- Calculate the aperture half-angle of the objective as
Approximate values for the refractive index are stated in Table I.
- Once the aperture half-angle is known, the
maximum plane-to-plane distance is calculated as
**Wide-field fluorescence:**=**d**_{z,max}**λ**_{em}**sin(α**)_{obj}**2***NA*_{obj}**(1 - cos(α**))_{obj}
**Transmission or phase microscopy:**=**d**_{z,max}**λ****sin(α**)_{max}**2***NA*_{max}**(1 - cos(α**))_{max}
**Confocal fluorescence:**=*d*_{z,max}**λ**_{eff}**sin(α**)_{obj}**, λ**=_{eff}**1****2****NA**_{obj}**(1 - cos(α**))_{obj}**1****+****1****λ**_{ex}**λ**_{em}
**Confocal two-photon fluorescence:**=**d**_{z,max}**λ**_{ex}**sin(α**)_{obj}**4***NA*_{obj}**(1 - cos(α**))_{obj}*'* with λbeing the irradiating wavelength (usually in the infrared)._{ex}
**Reflection confocal:**=*d*_{z,max}**λ****4****n** For transmission or phase microscopy,*NA*_{max}and α_{max}denote the corresponding values of the greater of the condensor and objective numerical aperture. - Ensure that the distance between successive image
planes (
*Dz*) in object coordinates is below*d*._{z,max}
Focus Series Examples For the parameters of the wide-field fluorescence example just given, the appropriate spacing between successive image planes should be chosen to be below
with the aperture half angle being estimated from the NA (1.3) and the refraction index of oil (n - 1.516) :Accordingly, the maximum spacing for confocal fluorescence microscopy (parameters as given in Section III,B) is calculated as
IV. PITFALLS: CONTRAST AND SAMPLING Suppose the object and its image consist of a sine wave with a single spatial frequency (e.g., 200nm distance between two maxima). The sampling limit requires sampling at more than the double frequency (i.e., D < 100nm). If this image is sampled too close
to the limiting frequency, there may be a problem.
Depending on the exact position of the object, one may
be fortunate enough to sample a maximum, then a
minimum, a maximum again, and so on or, if unlucky,
always sample half the maximum (see example in Fig.
4). In the latter case, the result would be indistinguishable
from an object with zero frequency. Even when
sampling at distances slightly below the required
minimum distance, a very low contrast can result for
images of small size. Therefore, one should oversample,
such that at least one full period of the resulting envelope
amplitude modulation is captured. In other words,
with _{xy}M pixels along a spatial direction, the stated
maximum sampling distance should be lowered by
multiplication with the factor M/(M + 1). For common
image sizes (e.g., 512 x 512), this factor is negligible, but
it can be substantial when acquiring only few Z sections
(in which case M is the number of sections).
A. When to Use UndersamplingFor some applications the sampling limit does not need to be obeyed. If, for example, the task is to count homogeneously filled cells with a fluorescent dye using a 20x, 0.9 NA objective, it makes a lot of sense to seriously undersample the data. For a cell to be identified it might suffice to detect two adjacent bright pixels. Thus the required sampling can, in some cases, depend on the size of the object structure to image. Undersampling (e.g., by binning) can reduce the amount of acquired data (important in screening applications), reduce the readout noise, dark current, and sometimes enlarge the field of view. In the mentioned application the introduced aliasing effects should be tolerable. In other applications (e.g., neuronal imaging of dendrites) it may be very tempting to undersample the data. However, thin structures (such as dendrites) may occasionally be lost in some pixels because they happen to lie between two sample points. Such missing gaps can then render a computer-based analysis difficult, as the structures appear ruptured and will also cause serious problems when successive computerized deconvolution is applied to data. In these cases it can even be better to reduce the numerical aperture of the objective than to seriously undersample at high NA.B. When to Use OversamplingFor some gray value-based image processing tasks, oversampling is recommendable (Young, 1996, 1988; Verbeek, 1985; Verbeek and vanVliet, 1993); e.g., if the aim is the precise determination of object positions, one should oversample data by a factor of at least 1.5 (sample at a 1.5 times smaller pixel pitch as compared to the limits given earlier). Simulations revealed (Heintzmann, 1999) that determination of the center of mass can result in a significant systematical error even when sampled according to the stated sampling limits. Furthermore, a smaller sampling distance determines more precisely where each photon hits the detector, thus leading to a slightly more precise estimate of the particle position (Heintzmann, 1999). Oversampling can also be useful when successive deconvolution of data is planned (see below). Note that with oversampling, the photon dose delivered to the sample can still be kept constant. Acquired images may look inferior at a first glance, but they still contain all the necessary information. Such data always allow for successive binning, resampling, and/or smoothing to enhance the visual appearance. C. Deconvolution: Out-of-Band
Reconstruction PossibilitiesWhen the computerized deconvolution is applied to data, it is often useful to oversample data during data acquisition. The reason is that constrained deconvolution is capable of "guessing" high-frequency components in the object structure that have not been acquired. This is enabled by the use of prior knowledge about the object, e.g., its positivity or smoothness (Sementilli et al., 1993). In principle, acquired data can
be resampled, but this always involves loss of information
about the photon statistics, or even interpolation
errors can result. Although software might be able
to reconstruct on a denser grid than raw data, there
may be (depending on the algorithm) some resampling
involved, which in turn can skew the statistics of
the deconvolution procedure, thus leading to inferior
results. As a rule of thumb, a two-fold oversampling
(two times smaller sampling distances as compared to
the given limits) should suffice even for advanced
deconvolution software.Appendix: Derivation of Cut-Off Frequencies To obtain the equations given in the text, an expression for the cut-off spatial frequency was derived and the Nyquist theorem applied, calculating the maximum sampling distance as half the distance corresponding to this cut-off spatial frequency. Note that this derivation is valid for high NA vector theory. Electric field components of a plane wave can be described by a single point in Fourier space. Its distance to the origin is proportional to the inverse wavelength. A lens forms its image by the constructive interference of converging parallel beams. It can thus be described by a "cap" (Gustafsson et al. 1995) in Fourier
space (Fig. 5a).
For incoherent fluorescence wide-field imaging, the intensity in focus describing the point spread function (PSF) is obtained as the square of the absolute magnitude of the electric field distribution. Because the optical transfer function (OTF) is the Fourier transformation of the PSF, it can be obtained as an autocorrelation of Fig. 5a, which is identical to the convolution with itself mirrored at the origin in Fourier space. The resulting region of support is depicted in Fig. 5b. The corresponding distances are the reciprocal values, which have to be halved, yielding the respective maximum sampling distance in XY and Z directions.
In the case of confocal microscopy, photons have to be excited and detected. This leads to a multiplication of the probabilities of excitation PSF (corresponding to the OTF in Fig. 5b, but for λ ) and the emission PSF
(corresponding to the OTF in Fig. 5b). In Fourier space,
this translates to a convolution of Fig. 5b as depicted in
Fig. 6, yielding the appropriate equations for the confocal
closed pinhole case. The size and shape of the
pinhole is described by a multiplicative modification
of the detection OTF with the Fourier transformed
pinhole. The two-photon (no detector pinhole) derivation
follows in a similar fashion, as its PSF is the square
of the excitation PSF, which can be described by autocorrelation
of Fig. 5b for λ_{ex}._{ex}When dealing with wide-field transmission, the situation is different. The image can no longer strictly be described as a convolution of the object with a point spread function. However, for incoherent imaging in transmission, such approximation still holds. Nevertheless it is preferable to think of this as a scattering problem. Neglecting multiple scattering, one can look for the scattering object vectors in Fourier space, which should be imaged by the system. Scattering theory states that the incoming wave vector plus the object vector yields the outgoing wave vector. Incoming and outgoing vectors are restricted by the Ewald sphere and the numerical aperture of the condensor and objective, respectively. As depicted in Fig. 7a the range of possible scattering vectors depends on both, the incoming light as defined by the numerical aperture of the condensor and the outgoing light as restricted by the objective. The equation for confocal reflection microscopy is obtained by considering the possible scattering vectors when illuminating and detecting through the same objective (Fig. 7b).
AcknowledgmentB. Rieger, S. H6ppner, E. Lemke, I. T. Young, and T. M. Jovin are thanked for their help in revising this manuscript and C. J. R. Sheppard for fruitful discussions on sampling. ReferencesGustafsson, M. G. L., Agard, D. A., and Sedat, J. W. (1995). Sevenfold improvement of axial resolution in 3D widefield microscopy using two objective lenses. Proc. SPIE 2412, 147-156. Heintzmann, R. (1999). "Resolution Improvement of Biological Light Microscopic, data." Ph.D. thesis, Institute of Applied Physics,
University of Heidelberg, Germany. Sementilli, P. J., Hunt, B. R., and Nadar, M. S. (1993). Analysis of the limit to superresolution in incoherent imaging. J. Opt. Soc. Am. A 10, 2265-2276. Sheppard, C. J. R. (1989). Axial resolution in confocal fluorescence microscopy. J. Microsc. 154, 237-241. Sheppard, C. J. R., Gu, M., Kawata Y., and Kawata, S. (1994). Threedimensional transfer functions for high-aperture systems. J. Opt.
Soc. Am. A 11, 593-598. Sheppard, C. J. R., Gan, X., Gu, M., and Roy, M. (1995). Signal-tonoise in confocal microscopes. In "'Handbook of Biological Confocal
Microscopy" (J. B. Pawley, ed.), pp. 363-371. Plenum Press, New
York. Verbeek, P. W. (1985). A class of sampling-error free measures in oversampled band-limited images. Pattern Recogn. Lett. 3,
287-292. Verbeek, P. W., and van Vliet, L. J. (1993). Estimators of 2D edge length and position, 3D surface area and position in sampled grey-valued images. Biolmaging 1, 47-61. Wilson, T. (1990). Confocal microscopy. In "Confocal Microscopy" (T.
Wilson, ed.), pp. 1-64. Academic Press, London. Young, I. T. (1985). Estimation of sampling errors. Cytometry 6,
273-274. Young, I. T. (1988). Sampling density and quantitative microscopy. Anal. Quant. Cytol. Histol. 10, 269-275. Young, I. T. (1996). Quantitative microscopy. IEEE Engineer. Med. Biol.
Mag. 15, 59-66. |

© 2018 Biocyclopedia | All rights reserved.