Search

This paper presents a time-resolved (TR) CMOS image sensor (CIS) using two-tap near-infrared (NIR) lock-in pixels with background light (BGL) cancelling capability for remote heart rate (HR) detection. Aimed to the application, the sensor features a 1024 (V) $\times $ 1280 (H) spatial resolution with 412 (V) $\times $ 1280 (H) region-of-interest-readout (ROI-readout) at 30 fps, a high sensitivity of 134.8 ke−/lux $\cdot $ s, and an ultra-low temporal random noise of 1.13 $\mathrm {e}_{\mathrm {rms}}^{\mathrm {-}}$ at analog gain of 64. A maximum charge modulation ratio of 90.1% is obtained for realizing lock-in operation by means of in-pixel lateral electric field charge modulator (LEFM). The remote HR detection is achieved by capturing the heart-beat-induced temporal variation of the oxy-hemoglobin (HbO2) concentration by the developed TR CIS with NIR lock-in technique, and larger than 98% detection accuracies are attained under both non-visible-light condition and fluctuant brightness ambient-light environments. This remarkable feature demonstrates the superiority of the proposed method for driver monitoring applications in particular.

A programmable sub-nanosecond time-gated four-tap CMOS lock-in pixel (LIP) image sensor that uses an in-pixel pulse generator (PG) is developed for time-resolved (TR) biomedical imaging applications, such as a fluorescence lifetime imaging microscopy (FLIM). We demonstrate the system’s effectiveness with simulations and measurements. Using pixel-level PG circuits, very narrow time windows (TWs) that are less than 0.8 ns can be used for charge modulation. This rapid gating allows very weak signals from living cells to be captured with a high signal-to-noise ratio (SNR). The prototype CMOS imaging system displays a fast intrinsic response of 170 ps at 472 nm and a very low temporal random noise of 0.85 e-rms at 45 frames/s under truly correlated-double-sampling operation using two-stage charge transfer. Four outputs from the LIPs allow real-time fluorescence lifetimes (FLTs) measurement, even from samples with multiple FLT components.

A large full well capacity (FWC) for wide signal detection range and low temporal random noise for high sensitivity lock-in pixel CMOS image sensor (CIS) embedded with two in-pixel storage diodes (SDs) has been developed and presented in this paper. For fast charge transfer from photodiode to SDs, a lateral electric field charge modulator (LEFM) is used for the developed lock-in pixel. As a result, the time-resolved CIS achieves a very large SD-FWC of approximately 7ke-, low temporal random noise of 1.2e-rms at 20 fps with true correlated double sampling operation and fast intrinsic response less than 500 ps at 635 nm. The proposed imager has an effective pixel array of ${\text{128}}{\text{(H)}}\times {\text{256}}{\text{(V)}}$ and a pixel size of ${\text{11.2}}\times {\text{11.2}}\, \mu {\text{m}}^{2}$ . The sensor chip is fabricated by Dongbu HiTek 1P4M 0.11 $\mu {\text{m}}$ CIS process.

A CMOS lock-in pixel image sensor with embedded storage diodes and lateral electric field modulation (LEFM) of photo-generated charge is developed for fluorescence lifetime imaging. The time-resolved CMOS image sensor (CIS) with two-tap lock-in pixels achieves a very high time resolution of 10 ps when images are averaged over 30 frames, a very short intrinsic response time of 180 ps at 374 nm, and a low temporal random noise of $1.75{\text{e}}^{-}_{\text{rms}}$ with true correlated double sampling (CDS) operation. In addition, by using the LEFM and optimized process, a very high extinction ratio of approximately 94% at 472 nm laser diode is achieved. The usefulness of the proposed CIS is demonstrated for fluorescence lifetime imaging with the simulation and measurement results.

A low temporal read noise and high conversion gain reset-gate-less CMOS image sensor (CIS) has been developed and demonstrated for the first time at photoelectron-counting-level imaging. To achieve a high pixel conversion gain without fine or special processes, the proposed pixel has two unique structures: 1) coupling capacitance between the transfer gate and floating diffusion (FD) and 2) coupling capacitance between the reset gate and FD, for removing parasitic capacitances around the FD node. As a result, a CIS with the proposed pixels is able to achieve a high pixel conversion gain of $220~\mu \text{V}/\text{e}^{ {{-}}}$ and a low read noise of 0.27e $^{-}_{\text {rms}}$ using correlated multiple-sampling-based readout circuitry.

A time-resolved CMOS image sensor (CIS) using two-tap near infrared (NIR) lock-in pixels with background light (BGL) cancelling capability for non-contact heart rate (HR) detection is presented. Aimed to the application, the sensor performances of $1.1\text{e}_{\text{rms}}^{-}$ temporal random noise and 90.1% maximum charge modulation ratio are achieved by using low noise readout circuit and in-pixel lateral electric field charge modulator (LEFM) techniques. The non-contact HR detection with accuracies of more than 98% is implemented by the developed sensor under both non-visible-light and fluctuant ambient-light brightness conditions. This remarkable performance indicates the superiority of the proposed technique for automotive security applications in particular.

We have developed and evaluated the large full well capacity (FWC) for wide signal detection range and low temporal noise for high sensitivity lock-in pixel CMOS image sensor (CIS) embedded with two storage-diodes (SDs). In addition, for fast charge transfer from photodiode (PD) to SDs, a lateral electric field charge modulator (LEFM) is used for the developed lock-in pixel. As a result, the time-resolved CIS achieves a very large FWC of approximately 7000e-, low temporal random noise of 1.17e-rms at 45fps with true correlated double sampling (CDS) operation, and fast intrinsic response less than 500ps at 635nm. The proposed imager has an effective pixel array of 128(H)×256(V) and a pixel size of 11.2×11.2μm2. The sensor chip is fabricated by a Dongbu HiTek 1P4M 0.11μm CIS process.

Recently, CMOS time-resolved imaging devices are being widely used for scientific and medical applications. A fluorescence lifetime imaging microscopy (FLIM), which is a powerful analysis tool in fundamental physics as well as in the life science, is a typical application for the time-resolved imaging devices. For better time-resolution in the lock-in pixel design, a multi-tap pixel architecture is very effective and useful. In this paper, we have proposed an 8-tap CMOS lock-in pixel with lateral electric field charge modulator (LEFM) and demonstrated the effectiveness of designed pixel by CAD simulation. The proposed pixel makes possible to measure the highly time-resolved images with a high signal to noise ratio (SNR) and to observe various images of cells even if a sample has a multi-lifetime component. An 8-tap time-resolved CMOS image sensor chip is developed by 0.11μm 1P4M CIS process technology.

Fluorescence-based time-resolved (TR) analysis techniques are fundamental and effective methods in life science and medicine. Among others, fluorescence lifetime imaging microscopy (FLIM) is one of the representative measurement techniques for biomedical applications. Recently, advanced all-solid-state imaging devices for FLIM, e.g., a CCD with optional electron-multiplication (EM) readout, a single-photon avalanche diode (SPAD), and a TR CMOS image sensor (CIS), have been reported [1–3]. These devices can be implemented compactly in comparison with the typical FLIM systems such as a time-correlated single-photon counting (TCSPC) system [4]. However, imaging devices based on the CCD or SPAD still have some issues, including the need for specialized fabrication processes, use of higher voltages, and greater circuit complexity. The recent TR CIS with two taps in a pixel [3] shows us a way to realize real-time lifetime imaging, but the two-tap CIS has a limitation that a sample's lifetime cannot be measured in real time if the sample has multi-exponential decay components.

In the past several years, CMOS image sensors (CISs) with sub-single-electron noise level, particularly, deep sub-electron read noise (less than 0.5e-rms), have been reported. Such an ultra-low noise level is realized with a reduced floating diffusion (FD) node capacitance for attaining the high pixel conversion gain (CG) [1,2], and a high-gain readout circuitry with noise-reduction capabilities [3,4]. Recently, a reset-gate-less (RGL) CMOS image sensor has been reported [5]. It shows an excellent read noise performance using an optimized pixel structure for high CG and high-gain column ADC with multiple sampling. In this technique, however, a very high pulsed voltage of approximately 25V for the FD reset is essential to cause a punch-through effect. It is not suitable for image sensors with high pixel resolution and high-speed signal readout.

An extremely low temporal noise and wide dynamic range CMOS image sensor is developed using low-noise transistors and high gray-scale resolution (17b) folding-integration/cyclic analog-to-digital converter (ADC). Two types of pixel are designed. One is a high conversion gain (HCG) pixel with removing the coupling capacitance between the transfer gate and the floating diffusion, and the other is a pixel for wide dynamic range (WDR) CMOS imager with a native transistor as a source follower amplifier. The CMOS image sensor that is in combination with the proposed pixels and the high performance column ADC has achieved a low pixel temporal noise of 1.1erms-, a wide dynamic range of 87.5 dB with the video rate operation (30 Hz) and the vertical fixed pattern noise of 1.08-μVrms. The implemented HCG CMOS imager and WDR CMOS imager using 0.18 μm technology have the pixel conversion gain of 73.2- and 22.8-μV/e-, respectively.

A 1.3-megapixel CMOS image sensor (CIS) with digital correlated double sampling and 17-b column-parallel two-stage folding-integration/cyclic analog-to-digital converters (ADCs) is developed. The image sensor has 0.021-erms- vertical fixed pattern noise, 1.2-erms- pixel temporal noise, and 85.0-dB dynamic range using 32 samplings in the folding-integration ADC mode. Despite the large number of samplings (32 times), the prototype image sensor is demonstrated at the video rate operation of 30 Hz by the new architecture of the proposed ADCs and the high-performance peripheral logic (or digital) parts using low-voltage differential signaling circuit. The developed 17-b CIS has no visible quantization noise at very low light level of 0.01 lx because of high grayscale resolution where 1LSB = 0.1-. The implemented CIS using 0.18- μm technology has the sensitivity of 20 V/lx ·s and the pixel conversion gain of 82 μV/e-.

A CMOS lock-in pixel image sensor with embedded storage diodes in a pixel and lateral electric field modulator (LEFM) is developed for time-resolved measurements such as a fluorescence lifetime imaging microscopy (FLIM) and near-infrared spectroscopy (NIRS). The time-resolved CMOS image sensor (CIS) with two-tap lock-in pixels achieves a very high time resolution of 10ps when images are averaged over 30 frames, a very short intrinsic response time of 180ps at 374nm, and a low temporal random noise of 1.75e-rms with true correlated double sampling (CDS) operation. In addition, by using the LEFM and optimized process, a very high extinction ratio of approximately 94% at 472nm laser diode is achieved. The usefulness of the proposed CIS is demonstrated by the device simulations and the measurements. Then, we confirmed that the developed imager is very suitable for the scientific applications such as a biomedical imaging and multifunctional cameras.

A CMOS image sensor with deep sub-electron read noise and high pixel conversion gain has been developed. Its performance is recognized through image outputs from an area image sensor, confirming the capability of photoelectroncounting- level imaging. To achieve high conversion gain, the proposed pixel has special structures to reduce the parasitic capacitances around FD node. As a result, the pixel conversion gain is increased due to the optimized FD node capacitance, and the noise performance is also improved by removing two noise sources from power supply. For the first time, high contrast images from the reset-gate-less CMOS image sensor, with less than 0.3e− rms noise level, have been generated at an extremely low light level of a few electrons per pixel. In addition, the photon-counting capability of the developed CMOS imager is demonstrated by a measurement, photoelectron-counting histogram (PCH).

In the work described in this paper, an image reproduction scheme with an ultra-high-speed temporally compressive multi-aperture CMOS image sensor was demonstrated. The sensor captures an object by compressing a sequence of images with focal-plane temporally random-coded shutters, followed by reconstruction of time-resolved images. Because signals are modulated pixel-by-pixel during capturing, the maximum frame rate is defined only by the charge transfer speed and can thus be higher than those of conventional ultra-high-speed cameras. The frame rate and optical efficiency of the multi-aperture scheme are discussed. To demonstrate the proposed imaging method, a 5×3 multi-aperture image sensor was fabricated. The average rising and falling times of the shutters were 1.53 ns and 1.69 ns, respectively. The maximum skew among the shutters was 3 ns. The sensor observed plasma emission by compressing it to 15 frames, and a series of 32 images at 200 Mfps was reconstructed. In the experiment, by correcting disparities and considering temporal pixel responses, artifacts in the reconstructed images were reduced. An improvement in PSNR from 25.8 dB to 30.8 dB was confirmed in simulations.

A low temporal noise and high dynamic range CMOS image sensor is developed. A 1Mpixel CMOS image sensor with column-parallel folding-integration and cyclic ADCs has 80μVrms (1.2e-) temporal noise, 82 dB dynamic range using 64 samplings in the folding-integration ADC mode. Very high variable gray-scale resolution of 13b through 19b is attained by changing the number of samplings of pixel outputs. The implemented CMOS image sensor using a 0.18-μm technology has the sensitivity of 10-V/lx·s, the conversion gain of 67- μV/e-, and linear digital code range of more than 4 decades.

A probabilistic method to remove the random telegraph signal (RTS) noise and to increase the signal level is proposed, and was verified by simulation based on measured real sensor noise. Although semi-photon-counting-level (SPCL) ultra-low noise complementary-metal-oxide-semiconductor (CMOS) image sensors (CISs) with high conversion gain pixels have emerged, they still suffer from huge RTS noise, which is inherent to the CISs. The proposed method utilizes a multi-aperture (MA) camera that is composed of multiple sets of an SPCL CIS and a moderately fast and compact imaging lens to emulate a very fast single lens. Due to the redundancy of the MA camera, the RTS noise is removed by the maximum likelihood estimation where noise characteristics are modeled by the probability density distribution. In the proposed method, the photon shot noise is also relatively reduced because of the averaging effect, where the pixel values of all the multiple apertures are considered. An extremely low-light condition that the maximum number of electrons per aperture was the only 2 e − was simulated. PSNRs of a test image for simple averaging, selective averaging (our previous method), and the proposed method were 11.92 dB, 11.61 dB, and 13.14 dB, respectively. The selective averaging, which can remove RTS noise, was worse than the simple averaging because it ignores the pixels with RTS noise and photon shot noise was less improved. The simulation results showed that the proposed method provided the best noise reduction performance.

Wide dynamic range active pixel sensor(APS) using a charge pump circuit has been designed by using 2-poly 4-metal standard CMOS technology. The structure of the proposed APS is similar to the structure of the conventional 3-Tr APS. The proposed unit pixel consists of one photodiode and three MOSFETs. Using a charge pump circuit, the dynamic range of the proposed APS is increased, compared to the conventional 3-Tr APS.

Fluorescence lifetime imaging microscopy (FLIM), which is a nondestructive and minimally invasive manner and can therefore be applied to living cells and tissues, is a great analysis tool in fundamental physics as well as in the life sciences. Charge-coupled devices (CCDs) [1] and single-photon avalanche diodes (SPADs) [2,3] are used for time-resolved lifetime measurement. In particular, SPAD-based time-resolved imagers have a high single-photon sensitivity and good noise robustness. However, they consist of a SPAD array with pixel circuitry, time-to-digital converters (TDCs), digital integrators to amplify signals, and readout circuitry. To implement the high photon-counting rate, a large number of TDCs and digital integrators are required. The spatial resolution of the SPAD-based time-resolved imagers is limited on this account. A recently reported time-resolved CMOS imager [4] using a draining-only modulation (DOM) technique has an attractive feature that a very simple pixel structure and 2-stage charge transfer without transfer gate (TG) can be simultaneously attained. However, it has a small aperture area, a comparatively low transfer speed, and multiple outputs are a challenge.

We have developed and evaluated the high responsivity and low dark leakage CMOS image sensor with the ring-gate shared-pixel design. A ring-gate shared-pixel design with a high fill factor makes it possible to achieve the low-light imaging. As eliminating the shallow trench isolation in the proposed pixel, the dark leakage current is significantly decreased because one of major dark leakage sources is removed. By sharing the in-pixel transistors such as a reset transistor, a select transistor, and a source follower amplifier, each pixel has a high fill-factor of 43 % and high sensitivity of 144.6 ke - /lx·sec. In addition, the effective number of transistors per pixel is 1.75. The proposed imager achieved the relatively low dark leakage current of about 104.5 e - /s (median at 60°C), corresponding to a dark current density J dark_proposed of about 30 pA/cm 2 . In contrast, the conventional type test pixel has a large dark leakage current of 2450 e - /s (median at 60°C), corresponding to J dark_conventional of about 700 pA/cm 2 . Both pixels have a same pixel size of 7.5×7.5 μm 2 and are fabricated in same process.

This paper presents a time-resolved CMOS image sensor with draining-only modulation (DOM) pixels for tube-less streak cameras. Although the conventional streak camera has high time resolution, the device requires high voltage and bulky system due to the structure with a vacuum tube. The proposed time-resolved imager with a simple optics realize a streak camera without any vacuum tubes. The proposed image sensor has DOM pixels, a delay-based pulse generator, and a readout circuitry. The delay-based pulse generator in combination with an in-pixel logic allows us to create and to provide a short gating clock to the pixel array. A prototype time-resolved CMOS image sensor with the proposed pixel is designed and implemented using 0.11um CMOS image sensor technology. The image array has 30(Vertical) x 128(Memory length) pixels with the pixel pitch of 22.4um. .

To observe molecular transport in a living cell, a high-speed CMOS image sensor for multi-point fluorescence correlation spectroscopy is developed. To achieve low-noise and high-speed simultaneously, a prototype CMOS image sensor is designed based on a complete pixel-parallel architecture and multi-channel pipelined pixel readout. The prototype chip with 10×10 effective pixels is fabricated in 0.18-μm CMOS image sensor technology. The pixel pitch and the photosensitive area are 56μm and 10μm in diameter without a microlens, respectively. In the experiment, the total sampling rate of 606kS/s is achieved. The measured average random noise is 24.9LSB, which is equivalent to about 2.5 electrons in average.

To achieve subnano-second temporal resolution imaging, we have proposed a compressive multi-aperture high-speed image sensor. We designed a CMOS image sensor with 5×3 apertures and 64×54 pixels aperture. Temporal resolution of 10ns was confirmed experimentally.

Multi-beam confocal microscopy without any physical pinhole was demonstrated. As a key device, a custom CMOS image sensor realizing a focal-plane pinhole array effect by special pixel addressing and discarding of the unwanted photocarriers was developed. The axial resolution in the confocal mode measured by FWHM for a planar mirror was 8.9 μm, which showed that the confocality has been achieved with the proposed CMOS image sensor.

This paper presents the design and implementation of a time-resolved CMOS image sensor with a high-speed lateral electric field modulation (LEFM) gating structure for time domain fluorescence lifetime measurement. Time-windowed signal charge can be transferred from a pinned photodiode (PPD) to a pinned storage diode (PSD) by turning on a pair of transfer gates, which are situated beside the channel. Unwanted signal charge can be drained from the PPD to the drain by turning on another pair of gates. The pixel array contains 512 (V) × 310 (H) pixels with 5.6 × 5.6 µm2 pixel size. The imager chip was fabricated using 0.11 µm CMOS image sensor process technology. The prototype sensor has a time response of 150 ps at 374 nm. The fill factor of the pixels is 5.6%. The usefulness of the prototype sensor is demonstrated for fluorescence lifetime imaging through simulation and measurement results.

An extremely low temporal noise and wide dynamic range CMOS image sensor is developed using low-noise transistors and high gray-scale resolution (17b) folding-integration/cyclic ADC. Two types of pixel are designed. One is a high conversion gain (HCG) pixel with removing the coupling capacitance between the transfer gate and the floating diffusion, and the other is a pixel for wide dynamic range (WDR) CMOS imager with the native transistor as a source follower amplifier. The CMOS image sensor which is in combination with the proposed pixels and the high performance column ADC has achieved a low pixel temporal noise of 1.1e− rms and a wide dynamic range of 87.5dB with the video rate operation (30Hz). In addition, the WDR pixel has a very small occurrence of the RTS noise because of the effect of the native transistor in the pixel. The implemented HCG CMOS imager and WDR CMOS imager using 0.18μm technology have the pixel conversion gain of 73.2-μV/e− and 22.8-μV/e−, respectively.

A CMOS imager for confocal multi-beam scanning microscopy, where the pixel itself works as a pinhole, is proposed. This CMOS imager is suitable for building compact, low-power, and confocal microscopes because the complex Nipkow disk with a precisely aligned pinhole array can be omitted. The CMOS imager is composed of an array of sub-imagers, and can detect multiple beams at the same time. To achieve a focal-plane pinhole effect, only one pixel in each subimager, which is at the conjugate position of a light spot, accumulates the photocurrent, and the other pixels are unread. This operation is achieved by 2-dimensional vertical and horizontal shift registers. The proposed CMOS imager for the confocal multi-beam scanning microscope system was fabricated in 0.18-μm standard CMOS technology with a pinned photodiode option. The total area of the chip is 5.0mm × 5.0mm. The number of effective pixels is 256(Horizontal) × 256(Vertical). The pixel array consists of 32(H) × 32(V) sub-imagers each of which has 8(H) × 8(V) pixels. The pixel is an ordinary 4-transistor active pixel sensor using a pinned photodiode and the pixel size is 7.5μm × 7.5μm with a fillfactor of 45%. The basic operations such as normal image acquisition and selective pixel readout were experimentally confirmed. The sensitivity and the pixel conversion gain were 25.9 ke-/lx•sec and 70 μV/e- respectively.

Low-noise CMOS image sensors (CIS) employing column-parallel amplifiers that significantly reduce temporal noise, as well as electron-multiplication CCD (EM-CCD) image sensors are becoming popular for very-low-light-level imaging. These low-noise imagers with high-gain amplification in either the charge or voltage domains sacrifice the intra-scene dynamic range. Scientific applications of solid-state imagers strongly require very low temporal noise and wide intra-scene dynamic range as well as very high gray-scale resolution. A column-parallel analog-to-digital converter (ADC) and column-level signal processing in CISs are key techniques to meet these requirements. Single-slope [1,2], successive-approximation [3] and cyclic ADCs [4] are widely used for the column-parallel ADC in CMOS imagers. However, these ADCs require additional gain enhancements to achieve very low temporal noise. A recently reported [5] delta-sigma (ΔΣ) ADC has an attractive feature that low temporal noise and high resolution can be simultaneously attained by an oversampling technique. However, for very high resolution, a high number of samplings per pixel output, e.g., more than 360 samplings for 16b, is required.

This paper describes a RTS (random telegraph signal) noise reduction technique for an active pixel CMOS image sensor (CIS) with in-pixel selectable dual source-follower amplifiers. In this CMOS image sensor, the lower-noise transistor in each pixel is selected in the readout operation using a table of determining the lower-noise transistors of all the pixels. A prototype image sensor with 65×290 pixels for demonstrating the effectiveness of this technique has been implemented using 0.18µm CMOS image sensor technology with pinned photodiode option. The measured result shows that the maximum noise using the amplifier-selection technique is reduced to 9.6e- from 17.2e- which is the maximum noise of the image array using one of two amplifiers in each pixel without selection.