Search

Smart cameras are among the emerging new fields of electronics. The points of interest are in the application areas, software and IC development. In order to reduce cost, it is worthwhile to invest in a single architecture that can be scaled for the various application areas in performance (and resulting power consumption). In this paper, we show that the combination of an SIMD (single-instruction multiple-data) processor and a general-purpose DSP is very advantageous for the image processing tasks encountered in smart cameras. While the SIMD processor gives the very high performance necessary by exploiting the inherent data parallelism found in the pixel crunching part of the algorithms, the DSP offers a friendly approach to the more complex tasks. The paper continues to motivate that SIMD processors have very convenient scaling properties in silicon, making the complete, SIMD-DSP architecture suitable for different application areas without changing the software suite. Analysis of the changes in power consumption due to scaling shows that for typical image processing tasks, it is beneficial to scale the SIMD processor to use the maximum level of parallelism available in the algorithm if the IC supply voltage can be lowered. If silicon cost is of importance, the parallelism of the processor should be scaled to just reach the desired performance given the speed of the silicon.

We explore the energy dissipation of the Linear Processor Array (LPA) as a function of the number of available resources (Processor Units P) within the array. This number P is an important parameter, as it reflects performance, relates parallel processing to energy dissipation, and influences the scaling of the various parts of the LPA architecture (memory, address generator, communication network). To make a comparison of the different design variants for a fixed datawidth possible, we propose a high-level energy dissipation model of the processor, which is based on a detailed analysis of a general convolution algorithm. It is shown that the energy dissipation of the LPA can roughly be described by the relationship Etotal ~ N/P with N presenting the datawidth in pixels. This relationship is derived from two observations: first, the largest contribution to Etotal is formed by the energy dissipated by the memories, and second, in our model of the LPA, the datawidth of the memories corresponds with the number of pixels N to be processed, which results in an increase of the access rate when P decreases. Furthermore, we have shown that the energy dissipation caused by communication within the LPA, increases with increasing number of resources: the trade-off between communication versus computation in parallel computing. This turns out to be negligible in the total energy dissipation, and we therefore conclude, that the optimum solution is found, when a full number of resources is applied within the LPA.

With the migration into automated driving for various classes of vehicles, affordable self-positioning upto at least cm accuracy is a goal to be achieved. Commonly used techniques such as GPS are either not accurate enough in their basic variant or accurate but too expensive. In addition, sufficient GPS coverage is in several cases not guaranteed. In this paper we propose positioning of a vehicle based on fusion of several sensor inputs. We consider inputs from improved GPS with internet based corrections, inertia sensors and vehicle sensors fused with computer vision based positioning. For vision-based positioning, cameras are used for feature-based visual odometry to do relative positioning and beacon-based for absolute positioning. Visual features are brought into a dynamic map which allows sharing information among vehicles and allows us to deal with less robust feautures. This paper does not present final results, yet it is intended to share ideas that are currently being investigated and implemented.

Hybrid video compression schemes such as MPEG2 and H.263 use an image memory for motion-compensated coding. In VLSI implementations, this image is usually stored in external RAM because of its large size. To reduce the overall system costs, we propose to compress the image by a factor of 4 to 5 before storage, which then enables embedding of the image memory on the encoder IC itself. The proposed encoder architecture remains in the DCT-domain, so motion estimation and compensation are now performed from this domain. To control and guarantee the actual storage, scalable compression is used. A hardware implementation is feasible and worthwhile compared to traditional encoders with no noticeable loss in performance.

The SPOT/VEGETATION mission is going to its end and ESA, CNES and industry investigated the possibility of proposing a "VEGETATION like instrument" onboard a satellite based on the PROBA concept and design. The studies have demonstrated that due to the recent advances in miniaturization technology, this objective can be reached without losing compromising quality with respect to the SPOT/VEGETATION Instrument. The resulting PROBA-V mission aims at ensuring continuity to SPOT/VEGETATION products, providing daily images at low resolution over the world. The instrument consists of Three-Mirror-Anastigmatic (TMA) cameras (to cover the 2250 km swath width), each equipped with blue, red, near-infrared and short-wave infrared detectors. As a passive thermal control method is used, a thermo-elastic model is used in the user segment on-ground to correct for the geometric and radiometric properties. While for this ESA project, Verhaert Space is responsible as prime for the instrument, VITO acts as prime for setting up and running the user segment, including geometric and radiometric calibration. Special attention is placed to dealing with the different noise characteristics, viewing angles and ground sampling distance of the detectors. The Ground Segment and the User Segment will exploit the satellite capabilities, receiving, processing, archiving and disseminating Primary products, for advanced users and corrected Synthesis for the common Vegetation end users at 1000m and 250m spatial resolution. However, in the end a more agile, area and power efficient satellite system is designed that gives equal and in many parameters better performance than the current SPOT/VEGETATION Instruments.

This paper presents Xetal-Pro SIMD processor, which is based on Xetal-II, one of the most computational-efficient (in terms of GOPS/Watt) processors available today. Xetal-Pro supports ultra wide V DD scaling from nominal supply to the sub-threshold region. Although aggressive V DD scaling causes severe throughput degradation, this can be compensated by the nature of massive parallelism in the Xetal family. The predecessor of Xetal-Pro, Xetal-II, includes a large on-chip frame memory (FM), which cannot operate reliably at ultra low voltage. Therefore we investigate both different FM realizations and memory organization alternatives. We propose a hybrid memory architecture which reduces the non-local memory traffic and enables further V DD scaling. Compared to Xetal-II operating at nominal voltage, we could gain more than 10× energy reduction while still delivering a sufficiently high throughput of 0.69 GOPS (counting multiply and add operations only). This work gives a new insight to the design of ultra-low energy SIMD processors, which are suitable for portable streaming applications.

This paper discusses a method for abnormal motion detection and its real-time implementation on a smart camera. Abnormal motion detection is a surveillance technique that only allows unfamiliar motion patterns to result in alarms. Our approach has two phases. First, normal motion is detected and the motion paths are trained, building up a model of normal behaviour. Feed-forward neural networks are here used for learning. Second, abnormal motion is detected by comparing the current observed motion to the stored model. A complete demonstration system is implemented to detect abnormal paths of persons moving in an indoor space. As platform we used a wireless smart camera system containing an SIMD (Single-Instruction Multiple-Data) processor for real-time detection of moving persons and an 8051 microcontroller for implementing the neural network. The 8051 also functions as camera host to broadcast abnormal events using ZigBee to a main network system.

We investigate compression of remote sensing images with a special geometry of non-square pixels. Two fundamentally different data reduction strategies are compared: a combination of pixel binning with near lossless compression and a method operating at higher compression rates. To measure the real impact of the compression, the image processing flow upto final products is included in the experiments. The effects on sensor non-uniformities and their corrections are explicitly modeled and measured. We conclude that it is preferable to apply higher compression rates than to rely on pixel binning, even if the derived images have lower resolutions.

Image processing is widely used in many applications, including medical imaging, industrial manufacturing and security systems. In these applications, the size of the image is often very large, the processing time should be very small and the real-time constraints should be met. Therefore, during the last decades, there has been an increasing demand to exploit parallelism in applications. It is possible to explore parallelism along three axes: data-level parallelism (DLP), instruction-level parallelism (ILP) and task-level parallelism (TLP). This paper explores the limitations and bottlenecks of increasing support for parallelism along the DLP and ILP axes in isolation and in combination. To scrutinize the effect of DLP and ILP in our architecture (template), an area model based on the number of ALUs (ILP) and the number of processing elements (DLP) in the template is defined, as well as a performance model. Based on these models and the template, a set of kernels of image processing applications has been studied to find Pareto optimal architectures in terms of area and number of cycles via multi-objective optimization.

The XETAL chip by Philips Electronics is a low-cost hardware-solution for image processing on pixel level. The architecture of XETAL focuses on a low-energy environment and it is therefore highly suited for integration into mobile vision and intelligent cameras. While hardware support for 2D-vision has reached the level of affordable state-of-the-art technology by thorough research, also real-time 3D-vision by stereo, based on the support by a low-cost and low-energy hardware, appears to be able to reach this level soon.

Single-Instruction Multiple-Data (SIMD) processing is valuable in numerous compute intensive application areas, especially in pixel processing on programmable processors. With SIMD, the nice thing is that the actual hardware can be scaled to the performance and the power consumption demands of the application domain, while the software suite remains equal. In this paper, we discuss the effect of scaling on power consumption, cost and performance which is related to the characteristics of applications ranging from mobile to medical video processing. Our analysis is based on experience obtained with the Philips Xetal SIMD processor [1,2].

A transpose switch matrix memory (TSMM) is proposed to enable a highly parallel single-chip CMOS sensor/image processor, Xetal, developed at Philips to perform JPEG compression at video rate (30 frames per second, fps) at an image dimension of 640/spl times/480 pixels. The integrated solution consists of 320 processing elements and 80 TSMMs, operates at 16 MHz clock rate and 3.3 V supply voltage, and is designed for fabrication at 0.25 micron technology. The processing system can sustain a maximum throughput of 5.12 billion operations per second consuming an estimated 120 mW providing a processing power efficiency of 7 BOPS/Watt. The Xetal architecture is capable of performing pixel level image processing such as fixed pattern noise (FPN) correction, defective pixel concealment, Bayer pattern filtering, RGB-YUV conversion, auto white balancing, and auto exposure control. The TSMM expands support to block level operations including chrominance subsampling, separable 8/spl times/8 recursive DCT, and ZZ scan required for JPEG.