Search

Multi-path interference causes depth errors in indirect time-of-flight (ToF) cameras. In this paper, resolving multi-path interference caused by surface reflections using a multi-tap macro-pixel computational CMOS image sensor is demonstrated. The imaging area is implemented by an array of macro-pixels composed of four subpixels embodied by a four-tap lateral electric field charge modulator (LEFM). This sensor can simultaneously acquire 16 images for different temporal shutters. This method can reproduce more than 16 images based on compressive sensing with multi-frequency shutters and sub-clock shifting. In simulations, an object was placed 16 m away from the sensor, and the depth of an interference object was varied from 1 to 32 m in 1 m steps. The two reflections were separated in two stages: coarse estimation based on a compressive sensing solver and refinement by a nonlinear search to investigate the potential of our sensor. Relative standard deviation (precision) and relative mean error (accuracy) were evaluated under the influence of photon shot noise. The proposed method was verified using a prototype multi-tap macro-pixel computational CMOS image sensor in single-path and dual-path situations. In the experiment, an acrylic plate was placed 1 m or 2 m and a mirror 9.3 m from the sensor.

This paper presents a time-of-flight (TOF) range imager with high-range resolution. The range imager employs a TOF measuring technique using an impulse photocurrent response and a high-speed lock-in pixel based on a draining-only modulation. To address a skew problem in a 2-D array of the pixels, a column-wise gating-clock skew calibration was implemented. Each pixel has an in-pixel buffer and decoupling capacitor to reduce row-wise skew. A test-implemented imager has $132 \times 120$ effective pixels. The skew calibration circuit successfully reduce the skew from 173 to 8 psrms. The range resolution was achieved to be 0.25 mm with a 32-mm range, which corresponds to a 1.7-ps time resolution.

This paper presents a CMOS time-of-flight (ToF) range image sensor using high-speed lock-in pixels with background light canceling capability. The proposed lock-in pixel uses MOS gate-induced lateral electric field control of depleted potential of pinned photodiode for implementing a multiple-tap charge modulator while achieving a high-speed charge transfer for high-time resolution. A TOF image sensor with 320 x 240 effective pixels is implemented using a 0.11- $\mu \text{m}$ CMOS image sensor process. The TOF sensor has a range resolution of less than 12 mm without background light and 20 mm under background line for the range from 0.8 to 1.8 m and integration time of 50 ms. The effectiveness of in-pixel background light canceling with a three-tap output pixel is demonstrated.

This paper presents a CMOS Time-of-Flight (TOF) range imager using pinned-photodiode based high-speed 4-tap lock-in pixels with lateral-electric-field charge modulators (LEFM) in a 0.11 um CIS process. The proposed lock-in pixel structure using lateral electric field control is suitable for implementing a multiple-tap charge modulator while achieving high-speed charge transfer for high time resolution. The TOF imager with the multiple-tap charge modulators is expected to have background light cancelling capability in one frame and to improve range resolution with the 4 time windows for a range-shifted light pulse. Measurement results of the implemented TOF imager with 160×240 pixels are reported.

In extremely low-light conditions, random telegraph signal (RTS) noise and dark current white defects become visible. In this paper, a multi-aperture imaging system and selective averaging method which removes the RTS noise and the dark current white defects by minimizing the synthetic sensor noise at every pixel is proposed. In the multi-aperture imaging system, a very small synthetic F-number which is much smaller than 1.0 is achieved by increasing optical gain with multiple lenses. It is verified by simulation that the effective noise normalized by optical gain in the peak of noise histogram is reduced from 1.38e- to 0.48e- in a 3 × 3-aperture system using low-noise CMOS image sensors based on folding-integration and cyclic column ADCs. In the experiment, a prototype 3 × 3-aperture camera, where each aperture has 200 × 200 pixels and an imaging lens with a focal length of 3.0 mm and F-number of 3.0, is developed. Under a low-light condition, in which the maximum average signal is 11e- per aperture, the RTS noise and dark current white defects are removed and the peak signal-to-noise ratio (PSNR) of the image is increased by 6.3 dB.

An extremely low temporal noise and wide dynamic range CMOS image sensor is developed using low-noise transistors and high gray-scale resolution (17b) folding-integration/cyclic analog-to-digital converter (ADC). Two types of pixel are designed. One is a high conversion gain (HCG) pixel with removing the coupling capacitance between the transfer gate and the floating diffusion, and the other is a pixel for wide dynamic range (WDR) CMOS imager with a native transistor as a source follower amplifier. The CMOS image sensor that is in combination with the proposed pixels and the high performance column ADC has achieved a low pixel temporal noise of 1.1erms-, a wide dynamic range of 87.5 dB with the video rate operation (30 Hz) and the vertical fixed pattern noise of 1.08-μVrms. The implemented HCG CMOS imager and WDR CMOS imager using 0.18 μm technology have the pixel conversion gain of 73.2- and 22.8-μV/e-, respectively.

A 1.3-megapixel CMOS image sensor (CIS) with digital correlated double sampling and 17-b column-parallel two-stage folding-integration/cyclic analog-to-digital converters (ADCs) is developed. The image sensor has 0.021-erms- vertical fixed pattern noise, 1.2-erms- pixel temporal noise, and 85.0-dB dynamic range using 32 samplings in the folding-integration ADC mode. Despite the large number of samplings (32 times), the prototype image sensor is demonstrated at the video rate operation of 30 Hz by the new architecture of the proposed ADCs and the high-performance peripheral logic (or digital) parts using low-voltage differential signaling circuit. The developed 17-b CIS has no visible quantization noise at very low light level of 0.01 lx because of high grayscale resolution where 1LSB = 0.1-. The implemented CIS using 0.18- μm technology has the sensitivity of 20 V/lx ·s and the pixel conversion gain of 82 μV/e-.

A CMOS image sensor using high-speed lock-in pixels for stimulated Raman scattering (SRS) spectroscopy is presented in this paper. The effective SRS signal from the stimulated emission of SRS mechanism is very small in contrast to the offset of a probing laser source, which is in the ratio of 10-4 to 10-5. In order to extract this signal, the common offset component is removed, and the small difference component is sampled using switched-capacitor integrator with a fully differential amplifier. The sampling is performed over many integration cycles to achieve appropriate amplification. The lock-in pixels utilizes high-speed lateral electric field charge modulator (LEFM) to demodulate the SRS signal which is modulated at high-frequency of 20MHz. A prototype chip is implemented using 0.11μm CMOS image sensor technology.

In the work described in this paper, an image reproduction scheme with an ultra-high-speed temporally compressive multi-aperture CMOS image sensor was demonstrated. The sensor captures an object by compressing a sequence of images with focal-plane temporally random-coded shutters, followed by reconstruction of time-resolved images. Because signals are modulated pixel-by-pixel during capturing, the maximum frame rate is defined only by the charge transfer speed and can thus be higher than those of conventional ultra-high-speed cameras. The frame rate and optical efficiency of the multi-aperture scheme are discussed. To demonstrate the proposed imaging method, a 5×3 multi-aperture image sensor was fabricated. The average rising and falling times of the shutters were 1.53 ns and 1.69 ns, respectively. The maximum skew among the shutters was 3 ns. The sensor observed plasma emission by compressing it to 15 frames, and a series of 32 images at 200 Mfps was reconstructed. In the experiment, by correcting disparities and considering temporal pixel responses, artifacts in the reconstructed images were reduced. An improvement in PSNR from 25.8 dB to 30.8 dB was confirmed in simulations.

A low temporal noise and high dynamic range CMOS image sensor is developed. A 1Mpixel CMOS image sensor with column-parallel folding-integration and cyclic ADCs has 80μVrms (1.2e-) temporal noise, 82 dB dynamic range using 64 samplings in the folding-integration ADC mode. Very high variable gray-scale resolution of 13b through 19b is attained by changing the number of samplings of pixel outputs. The implemented CMOS image sensor using a 0.18-μm technology has the sensitivity of 10-V/lx·s, the conversion gain of 67- μV/e-, and linear digital code range of more than 4 decades.

A probabilistic method to remove the random telegraph signal (RTS) noise and to increase the signal level is proposed, and was verified by simulation based on measured real sensor noise. Although semi-photon-counting-level (SPCL) ultra-low noise complementary-metal-oxide-semiconductor (CMOS) image sensors (CISs) with high conversion gain pixels have emerged, they still suffer from huge RTS noise, which is inherent to the CISs. The proposed method utilizes a multi-aperture (MA) camera that is composed of multiple sets of an SPCL CIS and a moderately fast and compact imaging lens to emulate a very fast single lens. Due to the redundancy of the MA camera, the RTS noise is removed by the maximum likelihood estimation where noise characteristics are modeled by the probability density distribution. In the proposed method, the photon shot noise is also relatively reduced because of the averaging effect, where the pixel values of all the multiple apertures are considered. An extremely low-light condition that the maximum number of electrons per aperture was the only 2 e − was simulated. PSNRs of a test image for simple averaging, selective averaging (our previous method), and the proposed method were 11.92 dB, 11.61 dB, and 13.14 dB, respectively. The selective averaging, which can remove RTS noise, was worse than the simple averaging because it ignores the pixels with RTS noise and photon shot noise was less improved. The simulation results showed that the proposed method provided the best noise reduction performance.

To demonstrate the low-noise performance of the multi-aperture imaging system using a selective averaging method, an ultra-high-sensitivity multi-aperture color camera with 2×2 apertures is being developed. In low-light conditions, random telegraph signal (RTS) noise and dark current white defects become visible, which greatly degrades the quality of the image. To reduce these kinds of noise as well as to increase the number of incident photons, the multi-aperture imaging system composed of an array of lens and CMOS image sensor (CIS), and the selective averaging for minimizing the synthetic sensor noise at every pixel is utilized. It is verified by simulation that the effective noise at the peak of noise histogram is reduced from 1.44 e- to 0.73 e- in a 2×2-aperture system, where RTS noise and dark current white defects have been successfully removed. In this work, a prototype based on low-noise color sensors with 1280×1024 pixels fabricated in 0.18um CIS technology is considered. The pixel pitch is 7.1μm×7.1μm. The noise of the sensor is around 1e- based on the folding-integration and cyclic column ADCs, and the low voltage differential signaling (LVDS) is used to improve the noise immunity. The synthetic F-number of the prototype is 0.6.

Ultra-high-speed cameras are a powerful tool for biology as well as physics and mechanics to analyze the process of ultra-high-speed phenomena. The frame rate of the state-of-the-art burst-readout ultra-high-speed silicon imagers has reached approximately 20Mfps [1,2]. To observe faster phenomena such as plasma generation in laser processing, the state of electrons in a chemical reaction, and so on, much faster cameras are desired. There are several factors that prevent the speed-up of the ultra-high-speed imager: high gate control voltages and high power dissipation for high-efficiency multi-stage charge transfer in CCD imagers, and the current density limit of the power and ground lines and RC-constant of the vertical readout lines in CMOS imagers. Computational imaging can be a promising option to break the design limit of solid-state ultra-high-speed imagers. Several dedicated CMOS imagers have been demonstrated [3,4]. This paper presents a demonstration of a single-chip ultra-high-speed multi-aperture CMOS imager based on compressive sampling. The imager performs single-shot burst-readout image acquisition at a frame rate of 200Mfps.

Fluorescence lifetime imaging microscopy (FLIM), which is a nondestructive and minimally invasive manner and can therefore be applied to living cells and tissues, is a great analysis tool in fundamental physics as well as in the life sciences. Charge-coupled devices (CCDs) [1] and single-photon avalanche diodes (SPADs) [2,3] are used for time-resolved lifetime measurement. In particular, SPAD-based time-resolved imagers have a high single-photon sensitivity and good noise robustness. However, they consist of a SPAD array with pixel circuitry, time-to-digital converters (TDCs), digital integrators to amplify signals, and readout circuitry. To implement the high photon-counting rate, a large number of TDCs and digital integrators are required. The spatial resolution of the SPAD-based time-resolved imagers is limited on this account. A recently reported time-resolved CMOS imager [4] using a draining-only modulation (DOM) technique has an attractive feature that a very simple pixel structure and 2-stage charge transfer without transfer gate (TG) can be simultaneously attained. However, it has a small aperture area, a comparatively low transfer speed, and multiple outputs are a challenge.

This paper presents an indirect time-of-flight (TOF) measurement technique with an impulse photocurrent response of a lock-in pixel. By using a short-pulse laser, the generated photocurrent can be presumed to be an impulse response. This facilitates the utilization of the full high-speed performance of the photodetector and gives high range resolution. As a proof-of-concept, a test chip with a lock-in pixel based on draining-only modulation was implemented using 0.11 μm CMOS image-sensor technology. The test chip achieved a range resolution of 0.29 mm in a 50-mm measurable range, which corresponds to a time resolution of 1.9 ps and the successful acquisition of a 3-mm example step.

In this paper, a lock-in pixel readout circuits using a high speed lateral electric field modulator with differential charge accumulation for a stimulated Raman scattering imager is presented. The stimulated Raman scattering works by detecting the vibrations in chemical bonds between atoms, and the generated Raman signal after stimulated scattering processes is very low compared to offset signal. By using a high-speed lateral electric field modulator with lock-in pixels differential charge accumulation technique using a sample-and hold circuit with a fully differential amplifier, very small effective Raman signal can be extracted from large offset signal and high dynamic range is achieved.

This paper presents a new ToF measurement technique using an impulse photocurrent response. In the proposed technique, a laser with a short pulse width for light source, which can be regarded as an impulse input for a detector. As a result, the range calculation is determined only by photocurrent response of the detector. A test chip fabricated in a 0.11um CIS technology employs a draining-only modulation pixel, which enables a high speed charge modulation. The measurable range measured to be 50 mm within nonlinear error of 5% and the average range resolution of 0.21 mm is achieved.

This paper presents a time-resolved CMOS image sensor with draining-only modulation (DOM) pixels for tube-less streak cameras. Although the conventional streak camera has high time resolution, the device requires high voltage and bulky system due to the structure with a vacuum tube. The proposed time-resolved imager with a simple optics realize a streak camera without any vacuum tubes. The proposed image sensor has DOM pixels, a delay-based pulse generator, and a readout circuitry. The delay-based pulse generator in combination with an in-pixel logic allows us to create and to provide a short gating clock to the pixel array. A prototype time-resolved CMOS image sensor with the proposed pixel is designed and implemented using 0.11um CMOS image sensor technology. The image array has 30(Vertical) x 128(Memory length) pixels with the pixel pitch of 22.4um. .

To observe molecular transport in a living cell, a high-speed CMOS image sensor for multi-point fluorescence correlation spectroscopy is developed. To achieve low-noise and high-speed simultaneously, a prototype CMOS image sensor is designed based on a complete pixel-parallel architecture and multi-channel pipelined pixel readout. The prototype chip with 10×10 effective pixels is fabricated in 0.18-μm CMOS image sensor technology. The pixel pitch and the photosensitive area are 56μm and 10μm in diameter without a microlens, respectively. In the experiment, the total sampling rate of 606kS/s is achieved. The measured average random noise is 24.9LSB, which is equivalent to about 2.5 electrons in average.

Time-of-Flight (ToF) range imagers have a wide range of applications, such as 3D mice, gesture-based remote controllers, amusement, robots, security systems, and automobiles. Numerous ToF range imager developments have been reported [1-4]. Recent developments are often based on CMOS image sensor technology with pinned photodiode options [5-7], which are suitable for cost-effective mass production. Reported CMOS ToF range imagers use single-tap or two-tap lock-in pixels; to cancel the influence of background light, two or four sub-frames are used to produce a background-canceled range image. These architectures, however, have difficulty with precise range measurements of moving objects, because background light cancelation is not guaranteed for moving objects. Lock-in pixels without any charge-draining gate suffer from background light during the readout time of the operation. Another important issue with CMOS ToF range imagers for high range resolution is the speed of lock-in pixels, which must be improved to use high-modulation-frequency light or short-duration light pulses.

Recently, 3D scanning systems have attracted rapidly rising attention in combination with 3D printers. One of the common technologies in contactless 3D scanners is the light-section method, which has advantages in term of accuracy. The method, however, requires a long base line between a camera and light source to achieve high resolution and a mechanical scanning system. A high range resolution Time-of-Flight (ToF) imager provides new possibilities of implementing a miniature head, which allows flexible scanning of an object with a complicated structure. The range resolution of reported CMOS ToF imagers [1-3] is limited to a few centimeters. For higher resolution, higher modulation frequency is required. However, the modulation frequency used for CMOS ToF imagers is limited to several tens of MHz. This paper presents a ToF imager with 0.3mm range resolution, which corresponds to 2ps time resolution. To achieve this high resolution, the imager uses a ToF measurement technique based on an impulse photocurrent response [4] and draining-only modulation (DOM) pixels [5]. To realize a range imager with 2D pixel array, column-wise gating-clock skew calibration is implemented to demonstrate simultaneous sub-mm ToF measurements for the whole pixel array.

To achieve subnano-second temporal resolution imaging, we have proposed a compressive multi-aperture high-speed image sensor. We designed a CMOS image sensor with 5×3 apertures and 64×54 pixels aperture. Temporal resolution of 10ns was confirmed experimentally.

Stimulated Raman scattering (SRS) imaging provides background-free chemical imaging with improved image contrast. Since the generated Raman signal is very low to be readout, a high-speed modulation lock-in pixel using differential charge accumulation and offset canceling to achieve high dynamic range is used.

An extremely low temporal noise and wide dynamic range CMOS image sensor is developed using low-noise transistors and high gray-scale resolution (17b) folding-integration/cyclic ADC. Two types of pixel are designed. One is a high conversion gain (HCG) pixel with removing the coupling capacitance between the transfer gate and the floating diffusion, and the other is a pixel for wide dynamic range (WDR) CMOS imager with the native transistor as a source follower amplifier. The CMOS image sensor which is in combination with the proposed pixels and the high performance column ADC has achieved a low pixel temporal noise of 1.1e− rms and a wide dynamic range of 87.5dB with the video rate operation (30Hz). In addition, the WDR pixel has a very small occurrence of the RTS noise because of the effect of the native transistor in the pixel. The implemented HCG CMOS imager and WDR CMOS imager using 0.18μm technology have the pixel conversion gain of 73.2-μV/e− and 22.8-μV/e−, respectively.

Fluorescence lifetime imaging is becoming a powerful tool in biology. A charge-domain CMOS Fluorescence Lifetime Imaging Microscopy (FLIM) chip using a pinned photo diode (PPD) and the pinned storage diode (PSD) with different depth of potential wells has been previously developed by the authors. However, a transfer gate between PPD and PSD causes charge transfer noise due to traps at the channel surface. This paper presents a time-resolved CMOS image sensor with draining only modulation pixels for fluorescence lifetime imaging, which removes the transfer gate between PPD and PSD. The time windowing is done by draining with a draining gate only, which is attached along the carrier path from PPD to PSD. This allows us to realize a trapping less charge transfer between PPD and PSD, leading to a very low-noise time-resolved signal detection. A video-rate CMOS FLIM chip has been fabricated using 0.18μm standard CMOS pinned diode image sensor process. The pixel consists of a PPD, a PSD, a charge draining gate (TD), a readout transfer gate (TX) between the PSD and the floating diffusion (FD), a reset transistor and a source follower amplifier transistor. The pixel array has 200(Row) x 256(Column) pixels and the pixel pitch is 7.5μm. Fundamental characteristics of the implemented CMOS FLIM chip are measured. The signal intensity of the PSD as a function of the TD gate voltage is also measured. The ratio of the signal for the TD off to the signal for the TD on is 212 : 1.

Low-noise CMOS image sensors (CIS) employing column-parallel amplifiers that significantly reduce temporal noise, as well as electron-multiplication CCD (EM-CCD) image sensors are becoming popular for very-low-light-level imaging. These low-noise imagers with high-gain amplification in either the charge or voltage domains sacrifice the intra-scene dynamic range. Scientific applications of solid-state imagers strongly require very low temporal noise and wide intra-scene dynamic range as well as very high gray-scale resolution. A column-parallel analog-to-digital converter (ADC) and column-level signal processing in CISs are key techniques to meet these requirements. Single-slope [1,2], successive-approximation [3] and cyclic ADCs [4] are widely used for the column-parallel ADC in CMOS imagers. However, these ADCs require additional gain enhancements to achieve very low temporal noise. A recently reported [5] delta-sigma (ΔΣ) ADC has an attractive feature that low temporal noise and high resolution can be simultaneously attained by an oversampling technique. However, for very high resolution, a high number of samplings per pixel output, e.g., more than 360 samplings for 16b, is required.