Search

This paper provides a review of design approaches towards artificial intelligence (AI) System-on-Chip. AI algorithms have progressed over the past decades from perceptron-based neural network (NN) and neuro-fuzzy (NF) system to today’s deep neural network (DNN) and neuromorphic computing. Recent DNN hardware accelerators focus on energy-efficient integration of digital circuits to realize real-time DNN operation while neuromorphic processors deploy new memory technologies with analog computation for low power consumption. However, different design approaches can be applied to such processor implementation with their pros and cons. This paper reviews from the early processor designs for NN and NF in both mixed-mode and digital implementations to the recent DNN SoC designs that we have proposed for a decade. The former content deals with NN and NF processors used as a functional building block of a machine vision SoC, while the latter concentrates on integration of the whole DNN function. We also provide a discussion on the approaches, and provide perspective on future research directions.

The power density limits of complementary metal–oxide–semiconductor (CMOS) technology could be overcome by moving from a binary to a ternary logic system. However, ternary devices are typically based on multi-threshold voltage schemes, which make the development of power-scalable and mass-producible ternary device platforms challenging. Here we report a wafer-scale and energy-efficient ternary CMOS technology. Our approach is based on a single threshold voltage and relies on a third voltage state created using an off-state constant current that originates from quantum-mechanical band-to-band tunnelling. This constant current can be scaled down to a sub-picoampere level under a low applied voltage of 0.5 V. Analysis of a ternary CMOS inverter illustrates the variation tolerance of the third intermediate output voltage state, and its symmetric in–out voltage-transfer characteristics allow integrated circuits with ternary logic and memory latch-cell functions to be demonstrated. Quantum-mechanical band-to-band tunnelling can be used to create an energy-efficient ternary logic technology that can be fabricated on the wafer scale using complementary metal–oxide–semiconductor (CMOS) processes.

Deep Neural Networks (DNNs) have become a promising solution to inject AI in our daily lives from self-driving cars, smartphones, games, drones, etc. In most cases, DNNs were accelerated by server equipped with numerous computing engines, e.g., GPU, but recent technology advance requires energy-efficient acceleration of DNNs as the modern applications moved down to mobile computing nodes. Therefore, Neural Processing Unit (NPU) architectures dedicated to energy-efficient DNN acceleration became essential. Despite the fact that training phase of DNN requires precise number representations, many researchers proved that utilizing smaller bit-precision is enough for inference with low-power consumption. This led hardware architects to investigate energy-efficient NPU architectures with diverse HW-SW co-optimization schemes for inference. This chapter provides a review of several design examples of latest NPU architecture for DNN, mainly about inference engines. It also provides a discussion on the new architectural researches of neuromorphic computers and processing-in-memory architecture, and provides perspectives on the future research directions.

This paper introduces an energy-efficient design method for Deep Neural Network (DNN) accelerator. Although GPU is widely used for DNN acceleration, its huge power consumption limits practical usage on mobile devices. Recent DNN accelerators are dedicated to high energy-efficiency to realize real-time DNN acceleration with low power consumption. But a hardware-oriented algorithm is essential for realistic implementation. Therefore, various techniques of network compression are applied with the DNN accelerators that utilize several schemes to reduce computational complexity in trade of accuracy loss. This work studies the optimization schemes and presents a DNN accelerator architecture by hardware-software co-optimization.

Despite their superiority in silicon integration, ring-oscillator-based digital PLLs (RO-DPLLs) are seldom used for mobile transceivers because they have difficulty in meeting key requirements, such as low phase noise (PN) and high-frequency resolution. Due to the dilemma of setting the optimal bandwidth, considering the ΔΣM noise and the ring DCO poor PN, conventional ΔΣM-based fractional-N RO-DPLLs are limited in their ability to achieve low PN. To address this issue, the use of a digital-to-time-converter (DTC) to cancel the quantization noise (Q-noise) has become a general trend [1]. Figure 17.3.1 shows that, using a DTC that generates τ DTC according to the control word of the DTC, D DCW , these DPLLs can have a wide bandwidth, thereby significantly suppressing the DCO PN. However, the problem is that any nonlinearity in the loop could cause a significant increase in fractional spurs. In practice, the DTC is the major source of this nonlinearity, so one solution is to improve its linearity by pre-distorting D DCW for its own characteristics, f DCW , [2], but this increases the design complexity. Another method is to use a successive requantizer (SR) as a quantizer (instead of a ΔΣM) [3]. The SR can mitigate fractional spurs despite the nonlinearity of the DTC, but the DTC must have a larger dynamic range than the ΔΣM with the same order.

A wearable electrical impedance tomography (EIT) system is proposed for the portable real-time 3-D lung ventilation monitoring. It consists of two types of SoCs, active electrode (AE)-SoC and Hub-SoC, mounted on wearable belts. The 48-channel AE-SoCs are integrated on flexible printed circuit board belt, and Hub-SoC is integrated in the hub module which performs data gathering and wireless communication between an external imaging device. To get high accuracy under the variation of conductivity, the dual-mode current stimulator provides the optimal frequency for time difference-EIT and frequency difference-EIT with simultaneous 4 k–128 kHz impedance sensing. A wide dynamic range instruments amplifier is proposed to provide 94 dB of wide dynamic range impedance sensing. In addition, the 48-channel AE system with the dedicated communication and calibration is implemented to achieve 1.4- $\text{m}\Omega $ sensitivity of impedance difference in the in vivo environment. The AE-/Hub-SoCs occupy 3.2 and 1.3 mm2 in 65-nm CMOS technology and consume $124~\mu \text{W}$ and 1.1 mW with 1.2 V supply, respectively. As a result, EIT images are reconstructed with 90% of accuracy, and up to 10 frames/s real-time 3-D lung images are successfully displayed.

An ultra-low power true random number generator (TRNG) based on a sub-ranging SAR analog-to-digital converter (ADC) is proposed. The proposed TRNG is composed of a coarse-SAR ADC with a low-power adaptive-reset comparator and a low-power dynamic amplifier. The coarse-ADC part is shared with a sub-ranging SAR ADC for area reduction. The shared coarse-ADC not only plays the role of discrete-time chaotic circuit but also reduces the overall SAR ADC energy consumption by selectively activating the fine-SAR ADC. Also, the proposed dynamic residue amplifier consumes only 48 nW and the adaptive-reset comparator generates a chaotic map with only 6-nW consumption. The proposed TRNG core occupies 0.0045 mm2 in 0.18- $\mu \text{m}$ CMOS technology and consumes 82 nW at 270-kbps throughput with 0.6-V supply. It successfully passes all of National Institute of Standards and Technology (NIST) tests, and it achieves the state-of-the-art figure-of-merit of 0.3 pJ/bit.

The advanced driver assistance system (ADAS) for adaptive cruise control and collision avoidance is strongly dependent upon the robust image recognition technology such as lane detection, vehicle/pedestrian detection, and traffic sign recognition. However, the conventional ADAS cannot realize more advanced collision evasion in real environments due to the absence of intelligent vehicle/pedestrian behavior analysis. Moreover, accurate distance estimation is essential in ADAS applications and semiglobal matching (SGM) is most widely adopted for high accuracy, but its system-on-chip (SoC) implementation is difficult due to the massive external memory bandwidth. In this paper, an ADAS SoC with behavior analysis with Artificial Intelligence functions and hardware implementation of SGM is proposed. The proposed SoC has dual-mode operations of high-performance operation for intelligent ADAS with real-time SGM in D-Mode (d-mode) and ultralow-power operation for black box system in parking-mode. It features: 1) task-level pipelined SGM processor to reduce external memory bandwidth by 85.8%; 2) region-of-interest generation processor to reduce 86.2% of computation; 3) mixed-mode intention prediction engine for dual-mode intelligence; and 4) dynamic voltage and frequency scaling control to save 36.2% of power in d-mode. The proposed ADAS processor achieves 862 GOPS/W energy efficiency and 31.4-GOPS/mm2 area efficiency, which are 1.53 $\times $ and 1.75 $\times $ improvements than the state of the art, with 30 frames/s throughput under 720p stereo inputs.

A low-latency and low-power stereo matching accelerator is monolithically integrated with a CMOS image sensor (CIS) for mobile applications. To reduce the overall latency, focal-plane processing is adopted by using the proposed analog census transform circuit (ACTC), and the image readout is pipelined with the following stereo matching process. In addition, a novel focal-plane rectification pixel array (FRPA) merges the rectification with the image readout without any additional processing latency. For area-efficient pixel design, sparse rectification is proposed, and the image rectification is implemented with only two additional switches in each pixel. A stereo matching digital processor (SMDP) is integrated with the CIS for cost aggregation. We present the full design including the layout with a 65 nm CMOS process, and the FRPA, the ACTC, and the SMDP achieve 11.0 ms latency with complete stereo matching stages, which is suitable for a smooth user interface. As a result, the 2-chip stereo matching system dissipates $573.9~\mu \text {J}$ /frame and achieves 17% energy reduction compared to a previous stereo matching SoC.

A low-power object recognition (OR) system with intuitive gaze user interface (UI) is proposed for battery-powered smart glasses. For low-power gaze UI, we propose a low-power single-chip gaze estimation sensor, called gaze image sensor (GIS). In GIS, a novel column-parallel pupil edge detection circuit (PEDC) with new pupil edge detection algorithm XY pupil detection (XY-PD) is proposed which results in $2.9\times $ power reduction with $16\times $ larger resolution compared to previous work. Also, a logarithmic SIMD processor is proposed for robust pupil center estimation, $ error, with low-power floating-point implementation. For OR, low-power multicore OR processor (ORP) is implemented. In ORP, task-level pipeline with keypoint-level scoring is proposed to reduce the number of cores as well as the operating frequency of keypoint-matching processor (KMP) for low-power consumption. Also, dual-mode convolutional neural network processor (CNNP) is designed for fast tile selection without external memory accesses. In addition, a pipelined descriptor generation processor (DGP) with LUT-based nonlinear operation is newly proposed for low-power OR. Lastly, dynamic voltage and frequency scaling (DVFS) for dynamic power reduction in ORP is applied. Combining both of the GIS and ORP fabricated in 65 nm CMOS logic process, only 75 mW average power consumption is achieved with real-time OR performance, which is $1.2\!\times\text{and}\;4.4 \times $ lower power than the previously published work.

A low-energy 40/160 MHz dual-band full duplex body channel communication (BCC) transceiver and a 13.56 MHz R-C oscillator-based super-regenerative transceiver are integrated in 65 nm CMOS mixed mode process for both entertainment and healthcare applications. The on-chip R-C duplexer uses notch filters for full duplex communication with 40 Mb/s data rate and combined dual-band operation shows 80 Mb/s data rate with half duplex communication. 40 MHz sine wave and 160 MHz rectangular wave are adopted for modulation in the dual-band transmitter with 30 dB SNR improvement, and shared-loop BPSK receiver reduces the power consumption by 25%. The proposed super-regenerative transceiver including an OOK transmitter and an R-C oscillator-based receiver achieves ${>}60\,\text{dB}$ interference rejection with 100 kb/s data rate and $42.5\,\upmu\text{W}$ power consumption under the 0.8 V supply.

Recently, 3D hand-gesture recognition (HGR) has become an important feature in smart mobile devices, such as head-mounted displays (HMDs) or smartphones for AR/VR applications. A 3D HGR system in Fig. 13.4.1 enables users to interact with virtual 3D objects using depth sensing and hand tracking. However, a previous 3D HGR system, such as Hololens [1], utilized a power consuming time-of-flight (ToF) depth sensor (>2W) limiting 3D HGR operation to less than 3 hours. Even though stereo matching was used instead of ToF for depth sensing with low power consumption [2], it could not provide interaction with virtual 3D objects because depth information was used only for hand segmentation. The HGR-based UI system in smart mobile devices, such as HMDs, must be low power consumption ( 1000 more MAC operations than previous feature-based stereo depth sensing, which is difficult in real-time with a mobile CPU, and therefore, a dedicated low-power CNN-based stereo matching SoC is required.

Approximate nearest neighbor searching has been studied as the keypoint matching algorithm for object recognition systems, and its hardware realization has reduced the external memory access which is the main bottleneck in object recognition process. However, external memory access reduction alone cannot satisfy the ever-increasing memory bandwidth requirement due to the rapid increase of the image resolution and frame rate of many recent applications such as advanced driver assistance system. In this paper, vocabulary forest (VF) processor is proposed that achieves both high accuracy and high speed by integrating on-chip database (DB) to remove external memory access. The area-efficient reusable-vocabulary tree architecture is proposed to reduce area, and the propagate-and-compute-array architecture is proposed to enhance the processing speed of the VF. The proposed VF processor can speed up the object matching stage by 16.4x compared with the state-of-the-art matching processor [Hong et al., Symp. VLSIC, 2013] for high resolution (Full-HD) and real-time (60 fps) video object recognition. It is fabricated using 65 nm CMOS technology and integrated into an object recognition SoC. The proposed VF chip achieves 2.07 M-vector/s throughput and 13.3 nJ/vector per-vector energy with 95.7% matching accuracy for 100 objects.

Augmented reality (AR) is being investigated in advanced displays for the augmentation of images in a real-world environment. Wearable systems, such as head-mounted display (HMD) systems, have attempted to support real-time AR as a next generation UI/UX [1-2], but have failed, due to their limited computing power. In a prior work, a chip with limited AR functionality was reported that could perform AR with the help of markers placed in the environment (usually 1D or 2D bar codes) [3]. However, for a seamless visual experience, 3D objects should be rendered directly on the natural video image without any markers. Unlike marker-based AR, markerless AR requires natural feature extraction, general object recognition, 3D reconstruction, and camera-pose estimation to be performed in parallel. For instance, markerless AR for a VGA input-test video consumes ~1.3W power at 0.2fps throughput, with TI's OMAP4430, which exceeds power limits for wearable devices. Consequently, there is a need for a high-performance energy-efficient markerless AR processor to realize a real-time AR system, especially for HMD applications.