Search

Bandpassed images, commonly used for edge detection, also retain information about intensities between the edge boundaries. Using the familiar Laplacian-of-Gaussian as a bandpass filter, we present a method to extract and code the edge-associated information (edge primitives) and recover an image representation with high structural fidelity. We demonstrate that the edge-primitives representation is compact and therefore can be coded with high compression ratios.

The quality of image reconstructions from discrete data suffers not only from the blurring of spatial detail caused by limitations in the spatial frequency response of electrooptical systems, but also from the aliasing generated if spatial detail has been undersampled. P. Mertz and F. Grey [Bell Syst. Tech. J. 13, 464 (1934)] and O. H. Schade [J. Soc. Motion Pict. Telev. Eng. 56, 131 (1955); 58, 181 (1952); 61, 97 (1953); 64, 593 (1955)] have observed that reasonable spot intensity profiles and photosensor aperture shapes of equivalent size result in about equal blurring but that some profiles and shapes suppress aliasing better than others. This paper presents quantitative results of the magnitude of aliasing and blurring as a function of random radiancefields typical for natural scenes and of spatial responses and sampling intervals typical for TV cameras and optical-mechanical scanners. These results indicate that aliasing may often be a larger source of degradation than either blurring or electronic noise.

A technique is presented for estimating spectral reflectance curves from multispectral image data even if the spectral samples are obtained from channels whose spectral responsivity is not narrowband. It is demonstrated that these reflectance estimates can be written as a linear combination of the spectral samples and that, analogous to Shannon's sampling theorem, if the spectral reflectance is a natural cubic spline, it can be estimated exactly provided the number of spectral channels is sufficiently large. Simulation results suggest that the accuracy of the spectral reflectance estimates is quite good and very insensitive to the spectral responsivity shapes.

Multiresponse imaging is a process that acquires Open image in new window images, each with a different optical response, and reassembles them into a single image with an improved resolution that can approach Open image in new window times the photodetector-array sampling lattice. Our goals are to optimize the performance of this process in terms of the resolution and fidelity of the restored image and to assess the amount of information required to do so. The theoretical approach is based on the extension of both image restoration and rate distortion theories from their traditional realm of signal processing to image processing which includes image gathering and display.

Shannon's mathematical theory of communication is extended to image gathering. Expressions are obtained for the total information that is received with a single image-gathering channel and with parallel channels. It is concluded that the aliased signal components carry information even though these components interfere with the within-passband components in conventional image gathering and restoration, thereby degrading the fidelity and visual quality of the restored image. An examination of the expression for minimum mean-square-error, or Wiener-matrix, restoration from parallel image-gathering channels reveals a method for unscrambling the within-passband and aliased signal components to restore spatial frequencies beyond the sampling passband out to the spatial frequency response cutoff of the optical aperture.

The fidelity and resolution of the traditional Wiener restorations given in the prevalent digital processing literature can be significantly improved when the transformations between the continuous and discrete representations in image gathering and display are accounted for. However, the visual quality of these improved restorations also is more sensitive to the defects caused by aliasing artifacts, colored noise, and ringing near sharp edges. In this paper, these visual defects are characterized, and methods for suppressing them are presented. It is demonstrated how the visual quality of fidelity-maximized images can be improved when (1) the image-gathering system is specifically designed to enhance the performance of the image-restoration algorithm, and (2) the Wiener filter is combined with interactive Gaussian smoothing, synthetic high edge enhancement, and nonlinear tone-scale transformation. The nonlinear transformation is used primarily to enhance the spatial details that are often obscurred when the normally wide dynamic range of natural radiance fields is compressed into the relatively narrow dynamic range of film and other displays.

This paper deals with the extension of information theory to the assessment of visual communication from scene to observer. The mathematical development rigorously unites the electro-optical design of image gathering and display devices with the digital processing algorithms for image coding and restoration. Results show that: end-to-end system analysis closely correlates with measurable and perceptual performance characteristics, such as data rate and image quality, respectively. The goal of producing the best possible image at the lowest data rate can be realized only if (a) the electro-optical design of the image-gathering device is optimized for the maximum-realizable information rate and (b) the image-restoration algorithm properly accounts for the perturbations in the visual communication channel.

Communication theory is used to establish a mathematical foundation for characterizing image systems that combine image gathering and display with digital processing to produce sharp, clear images consistently and economically. Characterization entails three elements: A mathematical model that accounts for image gathering, signal coding, and image restoration as a whole, from the scene to the observer. Figures of merit that account for the critical limiting factors that constrain the performance of image systems. Computational simulations that correlate quantitative assessments with measurable and perceptual performance. The pivotal figure of merit is the information rate in the image system. This rate is paired with the theoretical minimum data rate to assess the efficiency with which information about natural scenes can be conveyed, and it is paired with the maximum realizable fidelity to assess the quality with which images of these scenes can be restored. Quantitative assessments show that the design of the image-gathering device that is optimized for the highest realizable information rate corresponds closely, under appropriate conditions, to the design of the human eye. Consistent with this convergence of communication theory and evolution, it is intuitively attractive to find that the performance of the informationally optimized image system is highly robust. The Performance not only reaches the best that is possible for scenes with a priori prescribed statistical properties, and it also approaches the best that is possible for a wide range of other scenes, even for those that have periodic features or transients. Keywords: image system; image gathering; electro-optical design; insufficient sampling; signal coding; image restoration; image display; image quality; communication theory; information rate; data rate; entropy; fidelity; natural scenes; human eye; retinex transformation; wiener filter

This assessment of visual communication integrates the optical design of the image-gathering device with the digital processing for image coding and restoration. Results show that informationally optimized image gathering ordinarily can be relied upon to maximize the information efficiency of decorrelated data and the visual quality of optimally restored images.© (1994) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

We assess redundancy reduction in image coding in terms of the information acquired by the image-gathering process and the amount of data required to convey this information. A clear distinction is made between the theoretically minimum rate of data transmission, as measured by the entropy of the completely decorrelated data, and the actual rate of data transmission, as measured by the entropy of the encoded (incompletely decorrelated) data. It is shown that the information efficiency of the visual communication channel depends not only on the characteristics of the radiance field and the decorrelation algorithm, as is generally perceived, but also on the design of the image-gathering device, as is commonly ignored.

Multiresponse imaging is a process that acquires A images, each with a different optical response, and reassembles them into a single image with an improved resolution that can approach 1/sq rt A times the photodetector-array sampling lattice. Our goals are to optimize the performance of this process in terms of the resolution and fidelity of the restored image and to assess the amount of information required to do so. The theoretical approach is based on the extension of both image restoration and rate-distortion theories from their traditional realm of signal processing to image processing which includes image gathering and display.

A finer-than-sampling-lattice resolution image can be obtained using multiresponse image gathering and Wiener-matrix restoration. The multiresponse image gathering weighs the within-passband and aliased signal components differently, allowing the Wiener-matrix restoration filter to unscramble these signal components and restore spatial frequencies beyond the sampling passband of the photodetector array. A multiresponse images can be reassembled into a single minimum mean square error image with a resolution that is sq rt A times finer than the photodetector-array sampling lattice.

Object structure is one of the most important features for many imaging applications. In many applications in space recording the spatial structure adequately is a challenge due to the the wide range of illumination conditions encountered. Moreover communication constraints often limit the amount of data that can be transmitted. Motivated by these concerns we have developed a coding scheme which is robust to the variations in illumination conditions preserves high structural fidelity and provides high compression ratios. The high correlation between the original and decoded images demonstrates the potential of this coding scheme for machine vision applications.© (1991) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

The end-to-end performance of image gathering, coding, and restoration as a whole is considered. This approach is based on the pivotal relationship that exists between the spectral information density of the transmitted signal and the restorability of images from this signal. The information-theoretical assessment accounts for (1) the information density and efficiency of the acquired signal as a function of the image-gathering system design and the radiance-field statistics, and (2) the improvement in information efficiency and data compression that can be gained by combining image gathering with coding to reduce the signal redundancy and irrelevancy. It is concluded that images can be restored with better quality and from fewer data as the information efficiency of the data is increased. The restoration correctly explains the image gathering and coding processes and effectively suppresses the image-display degradations.

The process of image-gathering, coding, and restoration is presently treated in its entirety rather than as a catenation of isolated tasks, on the basis of the relationship between the spectral information density of a transmitted signal and the restorability of images from the signal. This 'information-theoretic' assessment accounts for the information density and efficiency of the acquired signal as a function of the image-gathering system's design and radiance-field statistics, as well as for the information efficiency and data compression that are obtainable through the combination of image gathering with coding to reduce signal redundancy. It is found that high information efficiency is achievable only through minimization of image-gathering degradation as well as signal redundancy.

Visual communication with retinex coding seeks to suppress the spatial variation of the irradiance (e.g., shadows) across natural scenes and preserve only the spatial detail and the reflectance (or the lightness) of the surface itself. The separation of reflectance from irradiance begins with nonlinear retinex coding that sharply and clearly enhances edges and preserves their contrast, and it ends with a Wiener filter that restores images from this edge and contrast information. An approximate small-signal model of image gathering with retinex coding is found to consist of the familiar difference-of-Gaussian bandpass filter and a locally adaptive automatic-gain control. A linear representation of this model is used to develop expressions within the small-signal constraint for the information rate and the theoretical minimum data rate of the retinex-coded signal and for the maximum-realizable fidelity of the images restored from this signal. Extensive computations and simulations demonstrate that predictions based on these figures of merit correlate closely with perceptual and measured performance. Hence these predictions can serve as a general guide for the design of visual communication channels that produce images with a visual quality that consistently approaches the best possible sharpness, clarity, and reflectance constancy, even for nonuniform irradiances. The suppression of shadows in the restored image is found to be constrained inherently more by the sharpness of their penumbra than by their depth.

Visual communication, in the form of telephotography and television, for example, can be regarded as efficient only if the amount of information that it conveys about the scene to the observer approaches the maximum possible and the associated cost approaches the minimum possible. Elsewhere we have addressed the problem of assessing the end to end performance of visual communication systems in terms of their efficiency in this sense by integrating the critical limiting factors that constrain image gathering into classical communications theory. We use this approach to assess the electro-optical design of image gathering devices as a function of the f number and apodization of the objective lens and the aperture size and sampling geometry of the phot-detection mechanism. Results show that an image gathering device that is designed to optimize information capacity performs similarly to the human eye. For both, the performance approaches the maximum possible, in terms of the efficiency with which the acquired information can be transmitted as decorrelated data, and the fidelity, sharpness, and clearity with which fine detail can be restored.

A method for extracting edge primitives from Mach-band patterns is presented together with a method for recovering image representations of features outlined by the edge boundaries. The accuracy, stability, and resolution of these representations are assessed. Since these representations are most commonly used in characterizing targets, this method of low-level processing offers new opportunities for computer vision and high data-compressing coding. Two bandpass filters are considered, the spatially invariant Laplacian of Gaussian filter and spatially variant intensity-dependent spatial (IDS) summation. It is shown that the recovery from the IDS bandpass data is particularly advantageous in applications for which robustness to local and temporal variations in illumination is important. It is concluded that the edge primitives extracted from bandpassed images can be an efficient way to store, transmit, and represent images.

To optimize edge detection with the familiar Laplacian-of-Gaussian operator, it has become common to implement this operator with a large digital convolution mask followed by some interpolation of the processed data to determine the zero crossings that locate edges. It is generally recognized that this large mask causes substantial blurring of fine detail. It is shown that the spatial detail can be improved by a factor of about four with either the Wiener-Laplacian-of-Gaussian filter or an image-plane processor. The Wiener-Laplacian-of-Gaussian filter minimizes the image-gathering degradations if the scene statistics are at least approximately known and also serves as an interpolator to determine the desired zero crossings directly. The image-plane processor forms the Laplacian-of-Gaussian response by properly combining the optical design of the image-gathering system with a minimal three-by-three lateral-inhibitory processing mask. This approach, which is suggested by Marr's model of early processing in human vision, also reduces data processing by about two orders of magnitude and data transmission by up to an order of magnitude.

Shannon's theory of information is used to optimize the optical design of sensor-array imaging systems which use neighborhood image-plane signal processing for enhancing edges and compressing dynamic range during image formation. The resultant edge-enhancement, or band-pass-filter, response is found to be very similar to that of human vision. Comparisons of traits in human vision with results from information theory suggest that: (1) Image-plane processing, like preprocessing in human vision, can improve visual information acquisition for pattern recognition when resolving power, sensitivity, and dynamic range are constrained. Improvements include reduced sensitivity to changes in light levels, reduced signal dynamic range, reduced data transmission and processing, and reduced aliasing and photosensor noise degradation. (2) Information content can be an appropriate figure of merit for optimizing the optical design of imaging systems when visual information is acquired for pattern recognition. The design trade-offs involve spatial response, sensitivity, and sampling interval.

Shannon's theory of information for communication channels is used to assess the performance of line-scan and sensor-array imaging systems and to optimize the design trade-offs involving sensitivity, spatial response, and sampling intervals. Formulations and computational evaluations account for spatial responses typical of line-scan and sensor-array mechanisms, lens diffraction and transmittance shading, defocus blur, and square and hexagonal sampling lattices.

In this paper a model of the processes involved in multispectral remote sensing and data classification is developed as a tool for designing and evaluating smart sensors. The model has both stochastic and deterministic elements and accounts for solar radiation, atmospheric radiative transfer, surface reflectance, sensor spectral reponses, and classification algorithms. Preliminary results are presented which indicate the validity and usefulness of this approach. Future capabilities of smart sensors will ultimately be limited by the accuracy with which multispectral remote sensing processes and their error sources can be computationally modeled.

Information theory is used to assess the performance of sensor-array imaging systems, with emphasis on the performance obtained with image-plane signal processing. By electronically controlling the spatial response of the imaging system, as suggested by the mechanism of human vision, it is possible to trade-off edge enhancement for sensitivity, increase dynamic range, and reduce data transmission. Computational results show that: signal information density varies little with large variations in the statistical properties of random radiance fields; most information (generally about 85 to 95 percent) is contained in the signal intensity transitions rather than levels; and performance is optimized when the OTF of the imaging system is nearly limited to the sampling passband to minimize aliasing at the cost of blurring, and the SNR is very high to permit the retrieval of small spatial detail from the extensively blurred signal. Shading the lens aperture transmittance to increase depth of field and using a regular hexagonal sensor-array instead of square lattice to decrease sensitivity to edge orientation also improves the signal information density up to about 30 percent at high SNRs.

A method is investigated for optimizing the end-to-end performance of image gathering and restoration for visual quality. To achieve this objective, one must inevitably confront the problems that the visual quality of restored images depends on perceptual rather than mathematical considerations and that these considerations vary with the target, the application, and the observer. The method adopted in this paper is to optimize image gathering informationally and to restore images interactively to obtain the visually preferred trade-off among fidelity resolution, sharpness, and clarity. The results demonstrate that this method leads to significant improvements in the visual quality obtained by the traditional digital processing methods. These traditional methods allow a significant loss of visual quality to occur because they treat the design of the image-gathering system and the formulation of the image-restoration algorithm as two separate tasks and fail to account for the transformations between the continuous and the discrete representations in image gathering and reconstruction.

An expression for the information capacity of the optical-mechanical line-scan imaging process is derived, which includes the effects of blurring of spatial detail, photosensor noise, aliasing, and quantization. Both the information capacity for a fixed data density and the information efficiency (i.e., the ratio of information capacity to data density) exhibit a distinct single maximum when displayed as a function of sampling rate, and the location of this maximum is determined by the system frequency response shape, SNR, and quantization interval.

In this paper we formulate and use information and fidelity criteria to assess image gathering and processing, combining optical design with image-forming and edge-detection algorithms. The optical design of the image-gathering system revolves around the relationship among sampling passband, spatial response, and signal-to-noise ratio (SNR). Our formulations of information, fidelity, and optimal (Wiener) restoration account for the insufficient sampling (i.e., aliasing) common in image gathering as well as for the blurring and noise that conventional formulations account for. Performance analyses and simulations for ordinary optical-design constraints and random scences indicate that (1) different image-forming algorithms prefer different optical designs; (2) informationally optimized designs maximize the robustness of optimal image restorations and lead to the highest-spatial-frequency channel (relative to the sampling passband) for which edge detection is reliable (if the SNR is sufficiently high); and (3) combining the informationally optimized design with a 3 by 3 lateral-inhibitory image-plane-processing algorithm leads to a spatial-response shape that approximates the optimal edge-detection response of (Marr's model of) human vision and thus reduces the data preprocessing and transmission required for machine vision.

Shannon's theory of information is used to optimize the optical design of sensor-array imaging systems which use neighborhood image-plane signal processing for enhancing edges and compressing dynamic range during image formation. The resultant edge-enhancement, or band-pass-filter, response is found to be very similar to that of human vision. Comparisons of traits in human vision with results from information theory suggest that: (1) Image-plane processing, like preprocessing in human vision, can improve visual information acquisition for pattern recognition when resolving power, sensitivity, and dynamic range are constrained. Improvements include reduced sensitivity to changes in lighter levels, reduced signal dynamic range, reduced data transmission and processing, and reduced aliasing and photosensor noise degradation. (2) Information content can be an appropriate figure of merit for optimizing the optical design of imaging systems when visual information is acquired for pattern recognition. The design trade-offs involve spatial response, sensitivity, and sampling interval.

Image gathering and processing are assessed in terms of information and fidelity, and the relationship between these two figures of merit is examined. It is assumed that the system is linear and isoplanatic and that the signal and noise amplitudes are Gaussian, wide-sense stationary, and statistically independent. Within these constraints, it is found that the combined process of image gathering and reconstruction (which is intended to reproduce the output of the image-gathering system) behaves as optical, or photographic, image formation in that the informationally optimized design of the image-gathering system ordinarily does not maximize the fidelity of the reconstructed image. The combined process of image gathering and restoration (which is intended to reproduce the input of the image-gathering system) behaves more as a communication channel in that the informationally optimized design of the image-gathering system tends to maximize the fidelity of optimally restored representations of the input.

Information theory is used to formulate a single figure of merit for assessing the performance of line-scan imaging systems as a function of their spatial response (PSF or MTF), sensitivity, and sampling and quantization intervals and of the statistical properties of a random radiance field. Information density and efficiency (i.e., the ratio of information density to data density) tend to be optimum when the MTF and sampling passband of the imaging system are matched to the Wiener spectrum of the radiance field. Computational results for the statistical properties of natural radiance fields and the responses of common line-scan imaging mechanisms indicate that information density and efficiency are not strongly sensitive to variations in typical statistical properties of the radiance field and that the best practically realizable performance is approached when the sampling intervals are approximately 0.5-0.7 times the equivalent diameter of the PSF.