Search

In this paper, we present the architecture of a smart imaging sensor (SIS) for face recognition, based on a custom-design smart pixel capable of computing local spatial gradients in the analog domain, and a digital coprocessor that performs image classification. The SIS uses spatial gradients to compute a lightweight version of local binary patterns (LBP), which we term ringed LBP (RLBP). Our face recognition method, which is based on Ahonen’s algorithm, operates in three stages: (1) it extracts local image features using RLBP, (2) it computes a feature vector using RLBP histograms, (3) it projects the vector onto a subspace that maximizes class separation and classifies the image using a nearest neighbor criterion. We designed the smart pixel using the TSMC 0.35 μm mixed-signal CMOS process, and evaluated its performance using postlayout parasitic extraction. We also designed and implemented the digital coprocessor on a Xilinx XC7Z020 field-programmable gate array. The smart pixel achieves a fill factor of 34% on the 0.35 μm process and 76% on a 0.18 μm process with 32 μm × 32 μm pixels. The pixel array operates at up to 556 frames per second. The digital coprocessor achieves 96.5% classification accuracy on a database of infrared face images, can classify a 150×80-pixel image in 94 μs, and consumes 71 mW of power.

This paper presents the architecture of a hardware accelerator for a cellular neural network (CeNN) with an application to real-time edge detection on visible-range and infrared video. The accelerator features fully-pipelined processing elements (PEs) that exploit the data parallelism in the algorithm to perform an iteration of the CeNN on a stream of video data with high throughput. The memory architecture exploits the locality of reference in the CeNN, so that each PE uses only 5 line buffers to store pixel, state, and output data, thus achieving low on-chip memory utilization. Implemented on a Xilinx XC7A200T FPGA running at 245MHz, the accelerator performs edge detection on 1080p video using a single CeNN iteration with a throughput of 118 frames per second (fps), a total latency of 15.7us, and 618mW of power consumption. The architecture features static reconfiguration to store built-in kernels and to add more PEs to support multiple iterations of the CeNN algorithm. More kernels can be added dynamically through a serial interface.

An intelligent readout integrated circuit (iROIC) capable of pixel-level local gradient operations is presented in this paper. Unlike conventional ROICs whose task is to simply read the pixels and send their raw outputs serially to a computer port, the proposed iROIC performs some essential tasks at pixel-level within the readout circuit, eliminating the need for raw data transmission/conversion/processing externally. The new iROIC uses a bidirectional integrator and a pair of switches to enable gradient operations within a pixel, with minimal overhead. As an enabler example, a real-time texture-based infrared face recognition algorithm is implemented utilizing the proposed iROIC with on-chip local gradient operations. The system successfully achieves a 96.95% correct hit rate over 636 infrared images of 53 individuals.

We present a striping noise compensation architecture for hyperspectral push-broom cameras, implemented on a Field-Programmable Gate Array (FPGA). The circuit is fast, compact, low power, and is capable of eliminating the striping noise in-line during the image acquisition process. The architecture implements a multi dimensional neural network (MDNN) algorithm for striping noise compensation previously reported by our group. The algorithm relies on the assumption that the amount of light impinging at the neighboring photo-detectors is approximately the same in the spatial and spectral dimensions. Under this assumption, two striping noise parameters are estimated using spatial and spectral information from the raw data. We implemented the circuit on a Xilinx ZYNQ XC7Z2010 FPGA and tested it with images obtained from a NIR N17E push-broom camera, with a frame rate of 25fps and a band-pixel rate of 1.888 MHz. The setup consists of a loop of 320 samples of 320 spatial lines and 236 spectral bands between 900 and 1700 nanometers, in laboratory condition, captured with a rigid push-broom controller. The noise compensation core can run at more than 100 MHZ and consumes less than 30mW of dynamic power, using less than 10% of the logic resources available on the chip. It also uses one of two ARM processors available on the FPGA for data acquisition and communication purposes.