Search

Abstract The cone photoreceptors in our eyes selectively transduce the natural light into spiking representations, which endows the brain with high energy-efficiency color vision. However, the cone-like device with color-selectivity and spike-encoding capability remains challenging. Here, we propose a metal oxide-based vertically integrated spiking cone photoreceptor array, which can directly transduce persistent lights into spike trains at a certain rate according to the input wavelengths. Such spiking cone photoreceptors have an ultralow power consumption of less than 400 picowatts per spike in visible light, which is very close to biological cones. In this work, lights with three wavelengths were exploited as pseudo-three-primary colors to form ‘colorful’ images for recognition tasks, and the device with the ability to discriminate mixed colors shows better accuracy. Our results would enable hardware spiking neural networks with biologically plausible visual perception and provide great potential for the development of dynamic vision sensors.

Textureless building surfaces composed of homogenized pixels could lead to failure of photometric consistency. However, textureless regions widely present in artificial scenes usually exhibit strong planarity enabling depth estimation of textureless regions with planar priors. However, existing methods for generating planar priors suffer from oversegmentation of large planes with textureless regions, which indicates that planarity is not fully exploited. In this study, we propose a novel generation method of planar prior by combining mean-shift clustering and superpixel segmentation. The planarity is fully utilized given preferential generation of planar priors for large planes with textureless regions in artificial scenes. Finally, a probabilistic graphical model is used to adopt the planar priors and smoothing constraints into depth estimation process. The image gradient is used as a criterion of the degree of texture to adaptively adjust the weights of different constraints. Experimental results on the benchmark dataset ETH3D, UDD5, and SenseFly demonstrate that the proposed method can effectively recover the depth information of textureless regions in high-resolution images to obtain highly complete three-dimensional (3-D) models of artificial scenes.

Multiview stereo (MVS) achieves efficient 3D reconstruction on Lambertian surfaces and strongly textured regions. However, the reconstruction of weakly textured regions, especially planar surfaces in weakly textured regions, still faces significant challenges due to the fuzzy matching problem of photometric consistency. In this paper, we propose a multiview stereo for recovering planar surfaces guided by confidence calculations, resulting in the construction of large-scale 3D models for high-resolution image scenes. Specifically, a confidence calculation method is proposed to express the reliability degree of plane hypothesis. It consists of multiview consistency and patch consistency, which characterize global contextual information and local spatial variation, respectively. Based on the confidence of plane hypothesis, the proposed plane supplementation generates new reliable plane hypotheses. The new planes are embedded in the confidence-driven depth estimation. In addition, an adaptive depth fusion approach is proposed to allow regions with insufficient visibility to be effectively fused into the dense point clouds. The experimental results illustrate that the proposed method can lead to a 3D model with competitive completeness and high accuracy compared with state-of-the-art methods.

Neuromorphic computing is a brain-inspired computing paradigm that aims to construct efficient, low-power, and adaptive computing systems by emulating the information processing mechanisms of biological neural systems. At the core of neuromorphic computing are neuromorphic devices that mimic the functions and dynamics of neurons and synapses, enabling the hardware implementation of artificial neural networks. Various types of neuromorphic devices have been proposed based on different physical mechanisms such as resistive switching devices and electric-double-layer transistors. These devices have demonstrated a range of neuromorphic functions such as multistate storage, spike-timing-dependent plasticity, dynamic filtering, etc. To achieve high performance neuromorphic computing systems, it is essential to fabricate neuromorphic devices compatible with the complementary metal oxide semiconductor (CMOS) manufacturing process. This improves the device’s reliability and stability and is favorable for achieving neuromorphic chips with higher integration density and low power consumption. This review summarizes CMOS-compatible neuromorphic devices and discusses their emulation of synaptic and neuronal functions as well as their applications in neuromorphic perception and computing. We highlight challenges and opportunities for further development of CMOS-compatible neuromorphic devices and systems.