Search

This paper introduces pattern-selective color image fusion and shows how it can be applied to two domains of image enhancement: extension of dynamic range and depth of focus. Pattern-selective fusion methods provide a mechanism for combining multiple monochromatic source images through identifying salient features in the source images at multiple scales and orientations, and combining those features into a single composite image result. In this paper, the pattern-selective fusion method is generalized into a framework that is equally applicable to monochrome, color, and multi-spectral imagery. This proposed fusion framework is then used to combine a set of color source images, taken from a sensor with varying aperture and focus settings, into a single fused image result that has improved dynamic range and depth of field over any of the other frames in the input sequence. Experimental results show the performance of the dynamic range and depth-of-field extension on imagery taken from consumer-grade video camera equipment.

This paper describes a family of reconfigurable processing elements (RPEs) used to support video processing for the Sarnoff Vision Front End 200 (VFE-200) vision system. Within the VFE-200 RPEs have been used to estimate visual motion, compute 3D scene structure using stereo analysis, perform geometric transformations (warps) on imagery with interpolation, and to act as triple ported frame store memory units. The RPEs described in this paper incorporate complex DRAM memory control interfaces, high precision fixed- and floating-point arithmetic (including floating point division), and sophisticated hybrids of memory and computational functions. Within this paper, the architecture and implementation of the RPEs and the VFE-200 are described, and examples of how the RPEs are used to support specific computer vision functions at real-time video rates are presented.

This paper introduces a method for extending the effective dynamic range and depth of focus of a sensor through pattern-selective color image fusion. Pattern-selective fusion methods provide a mechanism for combining multiple monochromatic source images through identifying salient features in the source images at multiple scales and orientations, and combining those features into a single fused image result. In this paper the pattern-selective fusion method is generalized into a framework that is equally applicable to monochrome, color and multi-spectral imagery. This proposed fusion framework is then used to combine a set of color source images, taken from a sensor with varying aperture and focus settings, into a single fused image result that has improved dynamic range and depth-of-field over any of the other frames in the input sequence. Experimental results show the performance of the dynamic range and depth-of-field extension on imagery taken from consumer-grade video camera equipment.

We describe a real-time system designed to construct a stable view of a scene through aligning images of an incoming video stream and dynamically constructing an image mosaic. This system uses a video processing unit developed by the David Sarnoff Research Center called the Vision Front End (VFE-100) for the pyramid-based image processing tasks required to implement this process. This paper includes a description of the multiresolution coarse-to-fine image registration strategy, the techniques used for mosaic construction, the implementation of this process on the VFE-100 system, and experimental results showing image mosaics constructed with the VFE-100. >

We use Sarnoff's VFE-200 next-generation video processor to demonstrate real-time algorithms for camera fixation, mosaic construction, and moving object detection and tracking from a moving camera. The algorithms are all derived from our ability to perform real-time image alignment. These capabilities are currently being used in several unmanned aerial and ground surveillance programs.

Two important problems in camera control are how to keep a moving camera fixated on a target point, and how to precisely aim a camera, whose approximate pose is known, towards a given 3D position. This paper describes how electronic image alignment techniques can be used to solve these problems, as well as provide other benefits such as stabilized video. Hence, stabilized, fixated imagery is obtained despite large latencies in the control loop, even for simple control strategies. These techniques have been tested using an airborne camera and real-time affine image alignment.

Presented is a new 80 GOPS video-processing chip capable of performing video rate vision applications. These applications include real-time video stabilization, mosaicking, video fusion, motion-stereo and video enhancement. The new vision chip, code-named Acadia, is the result of over 15 years of research and development by Sarnoff in the areas of multi-resolution pyramid-based vision processing and efficient computational architectures. The Acadia chip represents the third generation of ASIC technology developed by Sarnoff, and incorporates the processing functions found in Sarnoff's earlier PYR-1 and PYR-2 pyramid processing chips as well as numerous other functions found in Sarnoff-developed video processing systems, including the PVT200. A demonstration board is being implemented and includes two video decoders, a video encoder and a PCI interface.

This paper presents an image-processing algorithm for estimating both the egomotion of an outdoor robotic platform and the structure of the surrounding terrain. The algorithm is based on correlation, and is embedded in an iterative, multi-resolution framework. As such, it is suited to outdoor ground-based and underwater scenes. Both single-camera rigs and multiple-camera rigs can be accommodated. The use of multiple synchronized cameras results in more rapid convergence of the iterative approach. We describe how the algorithm operates and gives examples of its application in several robotic domains: Autonomous mobility of outdoor robots and Underwater robots.

We describe a novel technique for displaying video in which image frames are continually accumulated within the display devices to form an evolving mosaic. This `video mosaic' effectively extends the field of view and resolution of the source camera, and can aid in tasks such as surveillance and teleoperation.

The utility of 3D medical images has been limited by difficulties in analyzing and making measurements on the images. To tackle these problems, interactive workstation-based systems have been devised for visualizing and quantitating structures in 3D images. Unfortunately, these systems generally demand time-consuming, subjective, error-prone human interaction. To reduce the disadvantages of purely interactive techniques, some recent efforts have combined automatic image analysis with human interaction. These efforts demonstrate that for complex 3D medical applications, a judicious combination of human interaction and automatic computer-based processing is essential. We describe a system called INTERSEG (INTERactive SEGmentation) that combines human interaction and automatic processing for 3D radiological image analysis. INTERSEG is a graphical user interface system that can be used to: (1) construct interactively defined region cues; (2) invoke automatic image analysis; and (3) peruse/visualize analyzed images. The cues, which are easy to construct, convey problem-specific information and define the image-analysis task. The paper focuses on INTERSEG's capabilities for human interaction and visualization.