Search

Imaging in scattering media such as fog and water is important but challenging. Images suffer from poor visibility due to backscattering and signal attenuation. Most prior methods for scene recovery use active illumination scanners (structured and gated), which can be slow and cumbersome. On the other hand, natural illumination is inapplicable to dark environments. This paper addresses the need for a nonscanning recovery method which uses active scene irradiance. We study the formation of images under wide-field artificial illumination. Based on the formation model, this paper presents an approach for recovering the object signal. It also yields rough information about the 3D scene structure. The approach can work with compact simple hardware, having active wide-field polychromatic polarized illumination. The camera is fitted with a polarization analyzer. Two frames of the scene are instantly taken, with different states of the analyzer or light-source polarizer. A recovery algorithm follows the acquisition. It allows both the backscatter and the object reflection to be partially polarized. It thus unifies and generalizes prior polarization-based methods, which had assumed exclusive polarization of either of these components. The approach is limited to an effective range due to image noise and falloff of wide-field illumination. Thus, these limits and the noise sensitivity are analyzed. The approach particularly applies underwater. We therefore use the approach to demonstrate recovery of object signals and significant visibility enhancement in underwater field experiments. Index Terms--Computer vision, modeling and recovery of physical attributes, scene analysis--color, physics-based vision, vision in scattering media, inverse problems, polarization, image recovery.

Our global understanding of clouds and aerosols relies on the remote sensing of their optical, microphysical, and macrophysical properties using, in part, scattered solar radiation. These retrievals assume clouds and aerosols form plane-parallel, homogeneous layers and utilize 1D radiative transfer (RT) models, limiting the detail that can be retrieved about the 3D variability of cloud and aerosol fields and inducing biases in the retrieved properties for highly heterogeneous structures such as cumulus clouds and smoke plumes. To overcome these limitations, we introduce and validate an algorithm for retrieving the 3D optical or microphysical properties of atmospheric particles using multi-angle, multi-pixel radiances and a 3D RT model. The retrieval software, which we have made publicly available, is called Atmospheric Tomography with 3D Radiative Transfer (AT3D). It uses an iterative, local optimization technique to solve a generalized least-squares problem and thereby find a best-fitting atmospheric state. The iterative retrieval uses a fast, approximate Jacobian calculation, which we have extended from Levis et al. (2020) to accommodate open as well as periodic horizontal boundary conditions (BC) and an improved treatment of non-black surfaces. We validated the accuracy of the approximate Jacobian calculation for derivatives with respect to both the 3D volume extinction coefficient and the parameters controlling the open horizontal boundary conditions across media with a range of optical depths and single scattering properties and find that it is highly accurate for a majority of cloud and aerosol fields over oceanic surfaces. Relative root-mean-square errors in the approximate Jacobian for 3D volume extinction coefficient in media with cloud-like single scattering properties increase from 2% to 12% as the Maximum Optical Depths (MOD) of the medium increases from 0.2 to 100.0 over surfaces with Lambertian albedos < 0.2. Over surfaces with albedos of 0.7, these errors increase to 20%. Errors in the approximate Jacobian for the optimization of open horizontal boundary conditions exceed 50% unless the plane-parallel media providing the boundary conditions are very optically thin (~0.1). We use the theory of linear inverse RT to provide insight into the physical processes that control the cloud tomography problem and identify its limitations, supported by numerical experiments. We show that the Jacobian matrix becomes increasing ill-posed as the optical size of the medium increases and the forward scattering peak of the phase function decreases. This suggests that tomographic retrievals of clouds will become increasingly difficult as clouds becoming optically thicker. Retrievals of asymptotically thick clouds will likely require other sources of information to be successful. In Part 2 of this study, we examine how the accuracy of the retrieved 3D volume extinction coefficient varies as the optical size of the target medium increases using synthetic data. We do this to explore how the increasing error in the approximate Jacobian and increasingly ill-posed nature of the inversion in the optically thick limit affects the retrieval. We develop a method to improve retrieval accuracy in the optically thick limit. We also assess the accuracy of retrieved optical depths and surface irradiances and compare them to retrievals using 1D radiative transfer. [ABSTRACT FROM AUTHOR]

When imaging in scattering media, visibility degrades as objects become more distant. Visibility can be significantly restored by computer vision methods that account for physical processes occurring during image formation. Nevertheless, such recovery is prone to noise amplification in pixels corresponding to distant objects, where the medium transmittance is low. We present an adaptive filtering approach that counters the above problems: While significantly improving visibility relative to raw images, it inhibits noise amplification. Essentially, the recovery formulation is regularized, where the regularization adapts to the spatially varying medium transmittance. Thus, this regularization does not blur close objects. We demonstrate the approach in atmospheric and underwater experiments, based on an automatic method for determining the medium transmittance. Index Terms--Color, polarization, vision in bad weather, inverse problems, dehazing.

Imaging of objects under variable lighting directions is an important and frequent practice in computer vision, machine vision, and image-based rendering. Methods for such imaging have traditionally used only a single light source per acquired image. They may result in images that are too dark and noisy, e.g., due to the need to avoid saturation of highlights. We introduce an approach that can significantly improve the quality of such images, in which multiple light sources illuminate the object simultaneously from different directions. These illumination-multiplexed frames are then computationally demultiplexed. The approach is useful for imaging dim objects, as well as objects having a specular reflection component. We give the optimal scheme by which lighting should be multiplexed to obtain the highest quality output, for signal-independent noise. The scheme is based on Hadamard codes. The consequences of imperfections such as stray light, saturation, and noisy illumination sources are then studied. In addition, the paper analyzes the implications of shot noise, which is signal-dependent, to Hadamard multiplexing. The approach facilitates practical lighting setups having high directional resolution. This is shown by a setup we devise, which is flexible, scalable, and programmable. We used it to demonstrate the benefit of multiplexing in experiments. Index Terms--Physics-based vision, image-based rendering, multiplexed illumination, Hadamard codes, photon noise.

We present an approach to image the polarization state of object points in a wide field of view, while enhancing the radiometric dynamic range of imaging systems by generalizing image mosaicing. The approach is biologically-inspired, as it emulates spatially varying polarization sensitivity of some animals. In our method, a spatially varying polarization and attenuation filter is rigidly attached to a camera. As the system moves, it senses each scene point multiple times, each time filtering it through a different filter polarizing angle, polarizance, and transmittance. Polarization is an additional dimension of the generalized mosaicing paradigm, which has recently yielded high dynamic range images and multispectral images in a wide field of view using other kinds of filters. The image acquisition is as easy as in traditional image mosaics. The computational algorithm can easily handle nonideal polarization filters (partial polarizers), variable exposures, and saturation in a single framework. The resulting mosaic represents the polarization state at each scene point. Using data acquired by this method, we demonstrate attenuation and enhancement of specular reflections and semireflection separation in an image mosaic. Index Terms--Mosaicing, color, image fusion, physics-based vision, illumination, transparent layers, biology-inspired.

We present an approach for easily removing the effects of haze from passively acquired images. Our approach is based on the fact that usually the natural illuminating light scattered by atmospheric particles (airlight) is partially polarized. Optical filtering alone cannot remove the haze effects, except in restricted situations. Our method, however, stems from physics-based analysis that works under a wide range of atmospheric and viewing conditions, even if the polarization is low. The approach does not rely on specific scattering models such as Rayleigh scattering and does not rely on the knowledge of illumination directions. It can be used with as few as two images taken through a polarizer at different orientations. As a byproduct, the method yields a range map of the scene, which enables scene rendering as if imaged from different viewpoints. It also yields information about the atmospheric particles. We present experimental results of complete dehazing of outdoor scenes, in far-from-ideal conditions for polarization filtering. We obtain a great improvement of scene contrast and correction of color. OCIS codes: 290.1310, 330.0330, 260.5430, 100.2000, 100.3020, 150.5670.

We present an approach to significantly enhance the spectral resolution of imaging systems by generalizing image mosaicing. A filter transmitting spatially varying spectral bands is rigidly attached to a camera. As the system moves, it senses each scene point multiple times, each time in a different spectral band. This is an additional dimension of the generalized mosaic paradigm, which has recently demonstrated yielding high radiometric dynamic range images in a wide field of view, using a spatially varying density filter. The resulting mosaic represents the spectrum at each scene point. The image acquisition is as easy as in traditional image mosaics. We derive an efficient scene sampling rate, and use a registration method that accommodates the spatially varying properties of the filter. Using the data acquired by this method, we demonstrate scene rendering under different simulated illumination spectra. We are also able to infer information about the scene illumination. The approach was tested using a standard 8-bit black/white video camera and a fixed spatially varying spectral (interference) filter. Index Terms--Multispectral, hyperspectral imaging, color balance, enhancement, image fusion, physics-based vision, panorama, mosaic, mosaicing, illumination, image-based rendering.

A method to derive the 3D cloud envelope and the cloud development velocity from high spatial and temporal resolution satellite imagery is presented. The CLOUD instrument of the recently proposed C3IEL mission lends itself well to observing at high spatial and temporal resolutions the development of convective cells. Space-borne visible cameras simultaneously image, under multiple view angles, the same surface domain, every 20 s over a time interval of 200 s. In this paper, we present a method for retrieving cloud development velocity from simulated multi-angular-high-resolution TOA radiance cloud fields. The latter are obtained via the radiative transfer model 3DMCPOL, for a deep convective cloud case generated via the atmospheric research model Meso-NH, and via the image renderer Mitsuba for a cumulus case generated via the atmospheric research model SAM. Matching cloud features are found between simulations via block matching. Image coordinates of tie points are mapped to spatial coordinates via 3D stereo reconstruction of the external cloud envelope for each acquisition. The accuracy of the retrieval of cloud topography is quantified in terms of RMSE and bias that are respectively, less than 25 m and 15 m for the horizontal components and less than 40 m and 25 m for the vertical component. The inter-acquisition 3D velocity is then derived for each pair of tie points separated by 20 s. An independent method based on optimizing the superposition of the cloud top, issued from the atmospheric research model, allows to obtain a ground estimate for the velocity from two consecutive acquisitions. The distribution of retrieved velocity and ground estimate exhibits small biases but significant discrepancy in terms of distribution width. Furthermore, the average velocities derived from the mean altitude from ground for a cluster of localized cloud features identified over several acquisitions, both in the simulated images and in the physical point cloud, are in good agreement. [ABSTRACT FROM AUTHOR]

For the first time, vertically resolved long-term lidar measurements of the aerosol distribution were taken in Haifa, Israel. The measurements were performed by a PollyXT multi-wavelength Raman and polarization lidar. The lidar was measuring continuously over a 2-year period from March 2017 to May 2019. The resulting data set is a series of manually evaluated lidar optical property profiles. To identify the aerosol types in the observed layers, a novel aerosol typing method developed at TROPOS is used. This method applies optimal estimation to a combination of the lidar-derived intensive aerosol properties to determine the statistically most-likely contribution per aerosol component in terms of relative volume. A case study that shows several elevated aerosol layers illustrates this method and shows e.g. that coarse dust particles are observed up to 5 km height over Israel. From the whole data set, the seasonal distribution of the observed aerosol components over Israel is derived. Throughout all seasons, and with the highest contributions in summer, autumn, and winter, coarse spherical particles like sea salt, due to the coastal site, but also hygroscopic grown continental aerosol that was transported over the Mediterranean Sea was observed. During spring, coarse non-spherical particles attributed to desert dust were the mostly observed particles. This is consistent with the distinct dust season in Spring in Israel. An automated time-height-resolved air mass source attribution method identifies the dust sources in the Saharan and the Arabian deserts. Fine mode spherical particles also contribute significantly to the observed aerosol mixture during the most seasons. These particles originate mainly from the industrial region at the bay of Haifa. [ABSTRACT FROM AUTHOR]