Search

Convolution neural networks (CNNs) have demonstrated superior capability in computer vision, speech recognition, autonomous driving, and so forth, which are opening up an artificial intelligence (AI) era. However, conventional CNNs require significant matrix computation and memory usage leading to power and memory issues for mobile deployment and embedded chips. On the algorithm side, the emerging binary neural networks (BNNs) promise portable intelligence by replacing the costly massive floating-point compute-and-accumulate operations with lightweight bit-wise XNOR and popcount operations. On the hardware side, the computing-in-memory (CIM) architectures developed by the non-volatile memory (NVM) present outstanding performance regarding high speed and good power efficiency. In this paper, we propose an NVM-based CIM architecture employing a Preset-XNOR operation in/with the spin–orbit torque magnetic random access memory (SOT-MRAM) to accelerate the computation of BNNs (PXNOR-BNN). PXNOR-BNN performs the XNOR operation of BNNs inside the computing-buffer array with only slight modifications of the peripheral circuits. Based on the layer evaluation results, PXNOR-BNN can achieve similar performance compared with the read-based SOT-MRAM counterpart. Finally, the end-to-end estimation demonstrates $12.3\times $ speedup compared with the baseline with 96.6-image/s/W throughput efficiency.

Emerging nonvolatile memories (NVMs), including resistive RAM (RRAM), phase-change memory (PCM), and magnetic RAM (MRAM), have opened up new pathways for Computing-In-Memory (CIM). Those NVM technologies can achieve energy-efficient computational operations with only minor modification of the peripheral circuits. Despite many advantages provided by computational NVMs, parallelism is not sufficiently explored in such CIM designs. To break through this limitation on performance gain, we propose a data-streaming design for the NVM-based CIM (e.g., DASM) by leveraging the underlying parallelism in the hardware. DASM benefits from the massive parallelism of data-streaming computing, reduction in data movement of the CIM, and the nonvolatility of memory arrays. Specifically, data streaming operations can be implemented with CIM bitwise operations in both read-out and write-in procedures. In addition, we use the multilevel power gating for the memory array and connections to further boost the performance. Finally, we study a case of inference process for the quantized deep-neural-network-based on the DASM design. DASM architecture achieves $47.8\times $ , $5.1\times $ , $2.1\times $ speedup compared to the NVIDIA Jetson TK1 embedded GPU board, Intel Xeon E5-2640 CPU, the state-of-the-art field-programmable gate array (FPGA) design, with much lower power consumption.

As CMOS technology approaches its scaling limit, emerging nonvolatile memory (NVM) technologies become promising alternatives to DRAM due to their low leakage power and better scalability. However, the nonvolatile main memory system suffers from a new security vulnerability. An attacker can readily access sensitive information on the memory, since the nonvolatility allows information to be retained for a long time even after the power is OFF. While real-time memory encryption during memory accesses with dedicated Advanced Encryption Standard (AES) engine is an effective solution for this vulnerability, it incurs runtime performance and energy overhead. Alternatively, in this paper, we propose a fast and efficient AES in-memory (AIM) implementation, to encrypt the whole/part of the memory only when it is necessary. Rather than adding extra processing elements to the cost-sensitive memory, we take advantage of NVM’s intrinsic logic operation capability to implement the AES algorithm. We leverage the benefits (large internal bandwidth and dramatic data movement reduction) offered by the in-memory computing architecture to address the challenges of the bandwidth intensive encryption application. Embracing the massive parallelism inside the memory, AIM outperforms existing mechanisms with higher throughput yet lower energy consumption. The experimental results show that compared with state-of-the-art AES engine running at 2.1 GHz, AIM speeds up the encryption process by $80\times $ for a 1-GB NVM.

The STT-MRAM technology is a promising candidate for future on-chip cache memory because of its high density, low standby power, and nonvolatility. As the technology node scales, especially under 40-nm technology node, STT-MRAM cell design becomes a key issue to approach low power consumption, high access performance, and desirable reliability. The conventional 1T-1 magnetic tunnel junction (MTJ) and 2T-2MTJ cell designs cannot address these challenges efficiently. In this paper, we propose a novel 3T-3MTJ cell structure using the advanced perpendicular MTJ (p-MTJ) technology. It can store 2 bits with three MTJs. The differential sensing technique can be used to read out the most significant bit as fast as the 2T-2MTJ design. The sensing latency of 2 bits within the same cell is almost the same as the sensing latency of the 1T-1MTJ cell design. Therefore, the 3T-3MTJ cell can have the advantages of both 2T-2MTJ and 1T-1MTJ cells. Circuit-level simulations show that the proposed 3T-3MTJ cell structure can achieve a desirable tradeoff between storage density, access performance, and energy consumption compared to the prior 1T-1MTJ and 2T-2MTJ cell structures. Additionally, we propose a novel adaptive cache design based on the 3T-3MTJ cell structure, which can work in different modes to satisfy various memory access demands from different applications. Architecture level simulations validate the effectiveness of the proposed cache design.

Spin-transfer torque magnetic RAM (STT-MRAM), which uses a magnetic tunnel junction to store binary data, is a promising memory technology. With many benefits, such as low leakage power, high density, high endurance, and nonvolatility, it has been explored as an SRAM replacement for cache design or a DRAM replacement for main memory. Meanwhile, along with the continuous shrinking of CMOS process technology, the bias temperature instability (BTI) effect has become a major reliability issue. Prior work has investigated the influence of the BTI effect on the SRAM sense amplifier, but no investigation has been done for the STT-MRAM sense amplifier. Therefore, this paper investigates the BTI effect on STT-MRAM sense amplifiers. We propose a majority-based technique and an alternative sensing technique to reduce circuit degradation. To further improve sensing delay, we propose using forward body bias (FBB) on an access transistor with a positive voltage. Extensive simulation results are done to show the effectiveness of the proposed techniques. The sensing delay for reading zeros and ones can be reduced by 10.61% and 4.35%, respectively, on average, with the majority-based technique. The sensing delay for reading zeros and ones can be reduced by 4.42% and 1.83%, respectively, on average, using the alternative sensing technique. The sensing delay for reading zeros and ones can be reduced by 15.37% and 6.25%, respectively, on average, by using both techniques simultaneously. When using the majority-based and alternative sensing techniques with the FBB technique, the sensing delay for reading zeros and ones can be improved by 29.93% and 57.67%, respectively, on average. We also analyze the BTI-induced degradation of a high-performance sense amplifier and a low power sense amplifier with the proposed techniques. The simulation results show that our proposed technique and simulation flow can be easily extended to other sense amplifiers.

The breakdown of Dennard scaling prevents us from powering all transistors simultaneously, leaving a large fraction of dark silicon. This crisis has led to innovative work on power-efficient core and memory architecture designs. However, the research for addressing dark silicon challenges with network-on-chip (NoC), which is a major contributor to the total chip power consumption, is largely unexplored. In this paper, we comprehensively examine the network power consumers and the drawbacks of the conventional power-gating techniques. To overcome the dark silicon issue from the NoC’s perspective, we propose DimNoC, a dim silicon scheme, which leverages recent drowsy SRAM design and spin-transfer torque RAM (STT-RAM) technology to replace pure SRAM-based NoC buffers. In particular, we propose two novel hybrid buffer architectures: 1) a hierarchical buffer architecture, which divides the input buffers into a set of levels with different power states and 2) a banked buffer architecture, which organizes the drowsy SRAM and the STT-RAM in different banks, and accesses them in an interleaved fashion to hide the long write latency of STT-RAM. In addition, our hybrid buffer design enables NoC data retention mechanism by storing packets in drowsy SRAM and nonvolatile STT-RAM in a lossless manner. Combined with flow control schemes, the NoC data retention mechanism can improve network performance and power simultaneously. Our experiments over real workloads show that DimNoC can achieve 30.9% network energy saving, 20.3% energy-delay product reduction, and 7.6% router area reduction compared with pure SRAM-based NoC design.

Reducing interconnect delay and power consumption has become a major concern in deep submicron designs. 3-D technologies have been proposed as a promising solution to mitigate interconnect problems. This paper examines the electrical characterization of vertical intertier connections such as through silicon via (TSV) and microbumps considering process variations and studies their timing impact on the circuit level. We first provide parasitic RC characteristics of intertier connections including TSV and microbumps and examine their delay. Then circuit simulation is performed to evaluate the timing impact of intertier connections.

As technology scales, the aging effect caused by negative bias temperature instability (NBTI) has become a major reliability concern. In the mean time, reducing leakage power remains to be one of the key design goals. Because both NBTI-induced circuit degradation and standby leakage power have a strong dependency on the input vectors, input vector control (IVC) technique could be adopted to reduce the leakage power and mitigate NBTI-induced degradation. The IVC technique, however, is ineffective for larger circuits. Consequently, in this paper, we propose two gate replacement algorithms [direct gate replacement (DGR) algorithm and divide and conquer-based gate replacement (DCBGR) algorithm], together with optimal input vector selection, to simultaneously reduce the leakage power and mitigate NBTI-induced degradation. Our experimental results on 23 benchmark circuits reveal the following. 1) Both DGR and DCBGR algorithms outperform pure IVC technique by 15%-30% with 5% delay relaxation for three different design goals: leakage power reduction only, NBTI mitigation only, and leakage/NBTI cooptimization. 2) The DCBGR algorithm leads to better optimization results and save on average more than 10 runtime compared to the DGR algorithm. 3) The area overhead for leakage reduction is much more than that for NBTI mitigation.

In this paper, we proposed a new adder design called variable-latency adder (VL-adder). This technique allows the adder to work at a lower supply voltage than that required by a conventional adder while maintaining the same throughput. The VL-adder design can be further modified to overcome the effects of negative bias temperature instability (NBTI) on circuit delay. By applying VL-adder concept to a 64-bit carry-select adder design, more than 40% energy saving is obtained when a similar throughput is maintained.

Based on the proposed reliability characterization model, reliability-aware and low-power design is illustrated for the first time as a design methodology to balance reliability enhancement and power reduction. Low-power and reliable SRAM cell design, reliable dynamic voltage scaling (DVS) algorithm design, and voltage island partitioning and floorplanning for reliable system-on-a-chip (SOC) design are demonstrated as case studies of this new design methodology.

As technology scales, interconnects have become a major performance bottleneck and a major source of power consumption for sub-micro integrated circuit (IC) chips. One promising option to mitigate the interconnect challenges is 3D ICs, in which a stack of multiple device layers are put together on the same chip. In this paper, we explore the architectural design of cache memories using 3D circuits. We present a delay and energy model 3D cache delay-energy estimation tool (3D-Cacti) to explore different 3D design options of partitioning a cache. The tool allows partitioning of a cache across different device layers at various levels of granularity. The tool has been validated by comparing its results with those obtained from circuit simulation of custom 3D layouts. We also explore the effects of various cache partitioning parameters and 3D technology parameters on delay and energy to demonstrate the utility of the tool.

We propose a new class of methods for VLIW code compression using variable-sized branch blocks with self-generating tables. Code compression traditionally works on fixed-sized blocks with its efficiency limited by their small size. A branch block, a series of instructions between two consecutive possible branch targets, provides larger blocks for code compression. We compare three methods for compressing branch blocks: table-based, Lempel-Ziv-Welch (LZW)-based and selective code compression. Our approaches are fully adaptive and generate the coding table on-the-fly during compression and decompression. When encountering a branch target, the coding table is cleared to ensure correctness. Decompression requires a simple table lookup and updates the coding table when necessary. When decoding sequentially, the table-based method produces 4 bytes per iteration while the LZW-based methods provide 8 bytes peak and 1.82 bytes average decompression bandwidth. Compared to Huffman's 1 byte and variable-to-fixed (V2F)'s 13-bit peak performance, our methods have higher decoding bandwidth and a comparable compression ratio. Parallel decompression could also be applied to our methods, which is more suitable for VLIW architectures.

Code size "bloating" in embedded very long instruction word (VLIW) processors is a major concern for embedded systems since memory is one of the most restricted resources. In this paper, we describe a code compression algorithm based on arithmetic coding, discuss how to design decompression architecture, and illustrate the tradeoffs between compression ratio and decompression overhead, by using different probability models. Experimental results for a VLIW embedded processor TMS320C6x show that compression ratios between 67% and 80% can be achieved, depending on the probability models used. A precache decompression unit design is implemented in TSMC 0.25 mum and a test chip is fabricated.

In embedded system design, memory is one of the most restricted resources, posing serious constraints on program size. Code compression has been used as a solution to reduce the code size for embedded systems. Lossless data compression techniques are used to compress instructions, which are then decompressed on-the-fly during execution. Previous work used fixed-to-variable coding algorithms that translate fixed-length bit sequences into variable-length bit sequences. In this paper, we present a class of code compression techniques called variable-to-fixed code compression (V2FCC), which uses variable-to-fixed coding schemes based on either Tunstall coding or arithmetic coding. Though the techniques are suitable for both reduced instruction set computer (RISC) and very long instruction word (VLIW) architectures, they favor VLIW architectures which require a high-bandwidth instruction prefetch mechanism to supply multiple operations per cycle, and fast decompression is critical to overcome the communication bottleneck between memory and CPU. Experimental results for a VLIW embedded processor TMS320C6x show that the compression ratios using memoryless V2FCC and Markov V2FCC are around 82.5% and 70%, respectively. Decompression unit designs for memoryless V2FCC and Markov V2FCC are implemented in TSMC 0.25-/spl mu/m technology.