Search

As deep learning applications often encounter accuracy degradation due to the distorted inputs from a variety of environmental conditions, training with personal data has become essential for the edge devices. Hence, training on edge by supporting a trainable deep learning accelerator has been actively studied. Nevertheless, previous research does not consider the fundamental datapath for training and the importance of retaining the high performance for inference tasks. In this work, we propose NeuroFlix, a deep neural network training accelerator supporting heterogeneous data-type of floating- and fixed-point for input operands. From two perspectives: 1)separate precision decision for each input data, 2)maintenance of high performance on inference, we configure the data with low-bit fixed-point of activation/weight and floating-point based error gradient securing up to half-precision. A novel MAC architecture is designed to compute low/high-precision modes for the different input combinations. By substituting a high-cost floating-point based addition to brick-level separate accumulations, we realize both area-efficient architecture and high throughput for low-precision computation. Consequently, NeuroFlix outperforms the previous architectures of state-of-the-art configurations proving its high efficiency in both training and inference. By also comparing with the off-the-shelf bfloat16-based accelerator, it achieves 1.2/2.0 of speedup/energy-efficiency at training and further enhancement of 3.6/4.5 at inference.

Modern processors often need to switch among different power states based on usage scenarios, energy availability, and thermal conditions. Dynamic voltage and frequency scaling (DVFS) is a commonly used power management strategy to trade off performance and energy. As transistor scaling is reaching its limit, the viable supply voltage range where DVFS can operate is shrinking, which limits its effectiveness. To extend the performance-energy trade-off capabilities in modern processors, this article proposes dynamic core scaling (DCS) that does not rely on voltage scaling. DCS dynamically adjusts the active superscalar datapath resources so that programs run at a given percentage of their maximum speed while minimizing energy consumption at the same time. Since DCS does not need voltage scaling, it can be combined with DVFS to achieve greater energy savings. To effectively manage performance-energy trade-offs using a combination of DCS and DVFS, this article proposes an oracle controller that demonstrates the optimal control strategy, and two practical controllers that are applicable in real implementations. Evaluations using an 8-way superscalar processor implemented in a 45-nm circuit show that DCS is more effective in performance-energy trade-offs than DVFS at the high performance end for a number of SPEC CPU2000 benchmarks. When used together with DVFS, DCS saves an additional 20 percent of a full-size core’s energy on average. At the minimum operating voltage, DVFS hits its limit, while DCS is still able to achieve an average of 46 percent further energy reduction.

This paper introduces a novel class of architectures that unify true random number generation and private-key cryptography by reusing the cryptographic core for both tasks. The unified architecture is well suited for low-cost constrained secure integrated systems, in view of the inherent area efficiency and the low design effort entailed by conventional automated design flows. Clock pulse over-stretching in pulsed latch clocking generates randomness by inducing metastability and jittered oscillations. Shannon confusion and diffusion in the cryptographic datapath enforce high entropy and robustness against variations. Conventional cryptographic operation is alternatively performed at moderate clock pulsewidths. A 40-nm CMOS testchip demonstrates the proposed unified architecture with a compact area of 0.43 $\cdot 10^{6}~F^{2}$ ( $F\,\,=$ minimum feature size), based on a SIMON cryptographic core. The true random number generator (TRNG) output shows cryptographic-grade quality without any calibration across dice, process (across two manufacturing lots), voltage, and temperature variations. Energy per encryption down to 0.25 pJ/bit is demonstrated. Unification of TRNG and the cryptographic core results in inherent data locality and obfuscation of key generation within logic, improving the resilience to physical attacks.

This paper proposes a flexible eight-mode high parallel Galois SIMD ASIP(Application Specific Instruction Set Processor). It supports parallel executions of Gold, Scrambling, CRC, CC, Turbo, RM, PSS, SSS encoding LFSR (linear feedback shift registers) algorithms with high performance and flexibility. It can perform also general bit processing and m-sequence. Our design is based on proposed table conversion and a datapath for unified eight-mode encoding. Based on 28 nm digital CMOS technology, the total area is 0.177mm2 and the clock frequency can be up to 1 GHz. The throughputs of Gold, Scrambling, CRC32, CRC24, CRC16, CRC8, CC, Turbo are 64Gb/s, 64Gb/s, 128Gb/s, 168Gb/s, 256Gb/s, 512Gb/s, 3×80Gb/s, and 72Gb /s, respectively.

Hyperdimensional computing (HDC) is a brain-inspired computing paradigm-based on high-dimensional holistic representations of vectors. It recently gained attention for embedded smart sensing due to its inherent error-resiliency and suitability to highly parallel hardware implementations. In this work, we propose a programmable all-digital CMOS implementation of a fully autonomous HDC accelerator for always-on classification in energy-constrained sensor nodes. By using energy-efficient standard cell memory (SCM), the design is easily cross-technology mappable. It achieves extremely low power, 5 $\mu \text{W}$ in typical applications, and an energy efficiency improvement over the state-of-the-art (SoA) digital architectures of up to $3\times $ in post-layout simulations for always-on wearable tasks such as Electromyography (EMG) hand gesture recognition. As part of the accelerator’s architecture, we introduce novel hardware-friendly embodiments of common HDC-algorithmic primitives, which results in $3.3\times $ technology scaled area reduction over the SoA, achieving the same accuracy levels in all examined targets. The proposed architecture also has a fully configurable datapath using microcode optimized for HDC stored on an integrated SCM-based configuration memory, making the design “general-purpose” in terms of HDC algorithm flexibility. This flexibility allows usage of the accelerator across novel HDC tasks, for instance, a newly designed HDC-algorithm for the task of ball bearing fault detection.

The advent of programmable network switch ASICs and recent developments on other programmable data planes (NPUs, FPGAs) drive the renewed interest in network data plane programmability. The P4 language has emerged as a strong candidate to describe a protocol independent datapath pipeline. With its supported architectures, the P4 language provides an excellent way to define the packet processing and forwarding behavior, while leaving other networking components such as the traffic management engine, to non-programmable fixed function elements, based on the capabilities of most programmable devices. However, network flexibility is essential to meet the Quality of Service (QoS) requirements of traffic flows. Thus, enabling programmable control for fixed-function elements like traffic management is crucial. Towards that end we propose the use of virtual queues in the P4 pipeline, investigate the application of virtual queue-based traffic management, and portability of the approach using different P4 programmable targets. Specifically, we focus on virtual queue based Active Queue Management (AQM) for congestion policing and meeting the latency targets of distinct network slices. The solution is compared to P4 built-in functionality for bandwidth management using meters, proving also that the additional dimensions of control are achieved without compromising the processing complexity of the solution.

For most Internet-of-Things (IoT) applications, embedded processors typically execute lightweight tasks such as sensing and communication. The typical IoT program senses some information and sends them via a channel, usually a wireless channel with an RF circuit. These IoT nodes often require a system with networking capabilities and a low-power harvester implementation. This brief presents a sub- $\mu \text{W}$ 8-bit processor which is suitable for such IoT applications. The processor implements the Open8 Instruction Set Architecture (ISA) with an 8-bit datapath and 16-bit bus addressing. The chip contains the processor and a 4-KB of Static Random-Access-Memory (SRAM), and is fabricated by the 65-nm Silicon-On-Thin-Box (SOTB) process. The SOTB process is one of the Fully-Depleted Silicon-On-Insulator (FD-SOI) technology. Hence, the ability to control biasing voltages is one of its key advantages to achieve low-power. The experimental results show that the power consumption at the reverse-body bias can reach down to 50-nW with 0.5-V supply voltage and 32-KHz operating clock frequency. The completed microcontroller consists of the Open8 processor, 32-KB of Read-Only-Memory (ROM), 4-KB of SRAM, Serial Peripheral Interface (SPI), SPI programmer, debug module, General-Purpose In-Outs (GPIOs), and UART. The system was tested using an XC7A100T Xilinx Field-Programmable Gate Array (FPGA); it yielded 1.8% of the total FPGA utilization.

We successfully demonstrated an 8-bit-wide, bit-parallel datapath composed of an arithmetic logic unit and register files for high-throughput oriented SFQ microprocessors based on a gate-level-pipeline structure. Achieving high-speed operation in the bit-parallel datapath is difficult because of feedback paths. We used concurrent-flow clocking and counter-flow clocking in combination to solve the timing problem at the feedback path in the datapath, and we optimized the number of JJs and pipeline stages in the register file for solving the timing issue. We designed the datapath with the cell library for the AIST 10 kA/cm $^2$ Advanced Process. The total number of pipeline stages, Josephson junctions, and circuit area of the designed datapath were 52, 18448, and 3.81 mm × 4.05 mm, respectively. We obtained a relatively wide bias margin of the designed datapath at the target clock frequency of 50 GHz, and it operated up to 64 GHz in on-chip high-speed testing.

Approximate computing is a striking approach to design area-efficient low-power datapath units for fault buoyant applications. This brief presents the design of a novel 4: 2 approximate compressor that generates no error in the carry signal. The proposed compressor is employed for partial product (PP) compression in two variants of Dadda multiplier to see its effectiveness in error-resilient image and signal processing applications. In the targeted multipliers, the approximate 4:2 compressor is used in the least n PP columns, while the exact counterpart is used in the remaining most significant columns, and hence the maximum error is precisely maintained within 2n. PP compression is performed in stages using the Wallace approach, and the final two rows of sum and carry signals are added using a ripple carry adder in the basic design. In the proposed multiplier design-2, we do not generate sum bits in the approximate part. However, the proposed error-tolerant compressor is used in appropriate columns to propagate carry to the least significant column in the exact part. Performance evaluations using Cadence Encounter with 90 nm application specific integrated circuit technology revealed that the proposed-full width (P-FW) and the proposed-truncated (P-Trun) approximate multipliers demonstrate 22.7% and 32.4% power-delay product reduction compared to the standard multiplier. Implementations of the proposed multipliers in signal and image processing applications revealed superior performance in terms of accuracy compared to prior similar approximate designs.

In this work, we proposed a novel Coordinate Rotation DIgital Computer (CORDIC) rotator algorithm that converges faster by performing radix-2,4 and 16 CORDIC iterations while maintaining the scale factor implicitly constant. A mixed-radix is used to achieve convergence faster to reduce the computational latency of the CORDIC algorithm. The main concern of the higher radix CORDIC algorithm is the compensation of a variable scale factor. To solve this problem, the Taylor series approximation of sine and cosine is proposed for a higher radix CORDIC algorithm to achieve the scaling-free rotation of the two-dimensional vector. The scaling-free rotation of the proposed CORDIC algorithm removes the read-only memory (ROM) needed to store scale factor of higher radix CORDIC algorithm. Further, the proposed CORDIC algorithm is designed in rotation mode and optimized by removing the Z datapath for the digital signal processing (DSP) applications for which the angle of rotation is known in advance. Finally, the multipath delay commutator (MDC) fast Fourier transform (FFT) algorithm is implemented with the proposed CORDIC algorithm based rotator on FPGA. The proposed design is compared with existing designs. In a comparison between the radix-16 CORDIC rotator based FFT implementation and our proposed implementation, it has been found out that implementation proposed in this article has used 17% fewer resources.

Fast Fourier Transform is one of the most efficient methods of performing computation in Digital signal processing blocks. These computations are basically performed by the inherent floating-point ...

High accuracy and ever-increasing computing power have made deep neural networks (DNNs) the algorithm of choice for various machine learning, computer vision, and image processing applications across the computing spectrum. To this end, Google developed the tensor processing unit (TPU) to accelerate the computationally intensive matrix multiplication operation of a DNN on its systolic array architecture. Faults manifested in the datapath of such a systolic array due to latent manufacturing defects or single-event effects may lead to functional safety (FuSa) violation. Although DNNs are known to resist minor perturbations with their inherent fault-tolerant characteristics, we show that the classification accuracy of the model plummets from 97.4% to 7.75% with a minimal fault rate of 0.0003% in the accelerator, implying catastrophic circumstances when deployed across mission-critical systems. Hence, to ensure FuSa of such accelerators, this article provides an extensive FuSa assessment of the accelerator exposed to faults in the datapath, by varying the network parameters, position, and characteristics of the induced error across multiple exhaustive data sets. Furthermore, we propose two novel strategies to obtain a diminutive set of functional test patterns to detect FuSa violation in a DNN accelerator. Our experimental results demonstrate that the obtained test sets can achieve an average of 92.63% (in some cases, up to 100%) fault coverage with cardinality as low as 0.1% of the entire test data set.

Softwarization of Network Functions (NFs) accelerates automated deployment and management of services on next-gen networks. Combining flexibility and high-performance is a vital requirement for Network Functions Virtualisation (NFV); however, many studies have demonstrated that containerization or virtualization of NFs severely degrades the fundamental efficiency of packet forwarding. Virtual network I/O, a mechanism of packet transferring between a guest and the host, has been seen as the performance bottleneck in the PVP (Physical-Virtual-Physical) datapath, and one of the main causes of this deterioration is packet copy between them. Various techniques, such as zero-copy, pass-through, and hardware offloading, have been examined to alleviate the performance overhead. However, existing designs and implementations incur pragmatic issues, such as compatibility, manageability, and insufficient quality of performance. We propose yet another design and implementation of zero-copy/pass-through acceleration (named IOVTee) to resolve real-world problems as well as to enhance the forwarding efficiency. IOVTee takes advantage of pre-processing of virtual switches with achieving zero-copy on the receive (Rx) path. The pluggable style of IOVTee for vhost-user (the de-facto virtual network I/O) enables our approach to be transparent to both containers/VMs and virtual switches. In this article, we explain the heart of IOVTee, a fully software-based Rx queue mapping mechanism (between physical and virtual) that enables a concept of Virtual DMA Write-through (to the NF). Our evaluation results showed that applying IOVTee to vhost-user drastically increased efficiency of packet forwarding in the PVP datapath (by 45% and 98% for traffic of 64-byte and 1514-byte packets respectively).

Processing-in-Memory (PIM) or Near-Data Accelerator (NDA) has been recently revisited to mitigate the issues of memory and power wall, mainly supported by the maturity of 3D-staking manufacturing technology, and the increasing demand for bandwidth and parallel data access in emerging processing-hungry applications. However, as these designs are naturally decoupled from main processors, at least three open issues must be tackled to allow the adoption of PIM: how to offload instructions from the host to NDAs, since many can be placed along memory; how to keep cache coherence between host and NDAs, and how to deal with the internal communication between different NDA units considering that NDAs can communicate to each other to better exploit their adoptions. In this work, we present an efficient design to solve these challenges. Based on the hybrid Host-Accelerator code, to provide fine-grain control, our design allows transparent offloading of NDA instructions directly from a host processor. Moreover, our design proposes a data coherence protocol, which includes an inclusion-policy agnostic cache coherence mechanism to share data between the host processor and the NDA units, transparently, and a protocol to allow communication between different NDA units. The proposed mechanism allows full exploitation of the experimented state-of-the-art design, achieving a speedup of up to 14.6× compared to a AVX architecture on PolyBench Suite, using, on average, 82% of the total time for processing and only 18% for the cache coherence and communication protocols.

As one of the most promising energy-efficient emerging paradigms for designing digital systems, approximate computing has attracted a significant attention in recent years. Applications utilizing approximate computing (AxC) can tolerate some loss of quality in the computed results for attaining high performance. Approximate arithmetic circuits have been extensively studied; however, their application at system level has not been extensively pursued. Furthermore, when approximate arithmetic circuits are applied at system level, error-accumulation effects and a convergence problem may occur in computation. Multiple approximate components can interact in a typical datapath, hence benefiting from each other. Many applications require more complex datapaths than a single multiplication. In this paper, a hardware/software co-design methodology for adaptive approximate computing is proposed. It makes use of feature constraints to guide the approximate computation at various accuracy levels in each iteration of the learning process in Artificial Neural Networks (ANNs). The proposed adaptive methodology also considers the input operand distribution and the hybrid approximation. Compared with a baseline design, the proposed method significantly reduces the power-delay product while incurring in only a small loss of accuracy. Simulation and a case study of image segmentation validate the effectiveness of the proposed methodology.

Due to the fast-growing number of connected tiny devices to the Internet of Things (IoT), providing end-to-end security is vital. Therefore, it is essential to design the cryptosystem based on the requirement of resource-constrained IoT devices. This article presents a lightweight advanced encryption standard (AES), a high-secure symmetric cryptography algorithm, implementation on field-programmable gate array (FPGA) and 65-nm technology for resource-constrained IoT devices. The proposed architecture includes 8-bit datapath and five main blocks. We design two specified register banks, Key-Register and State-Register, for storing the plain text, keys, and intermediate data. To reduce the area, Shift-Rows is embedded inside the State-Register. To adapt the Mix-Column to 8-bit datapath, we design an optimized 8-bit block for Mix-Columns with four internal registers, which accept 8-bit and send back 8-bit. Also, a shared optimized Sub-Bytes is employed for the key expansion phase and encryption phase. To optimize Sub-Bytes, we merge and simplify some parts of the Sub-Bytes. To reduce power consumption, we apply the clock gating technique to the design. Application-specific integrated circuit (ASIC) implementation results show a respective improvement in the area over the previous similar works from 35% to 2.4%. Based on the results, the proposed design is a suitable cryptosystem for tiny IoT devices.

Personalized recommendation systems are gaining significant traction due to their industrial importance. An important building block of recommendation systems consists of what is known as the embedding layers, which exhibit a highly memory-intensive characteristics. Fundamental primitives of embedding layers are the embedding vector gathers followed by vector reductions, which exhibit low arithmetic intensity and becomes bottlenecked by the memory throughput. To address this issue, recent proposals in this research space employ a near-data processing (NDP) solution at the DRAM rank-level, achieving a significant performance speedup. We observe that prior NDP solutions based on rank-level parallelism leave significant performance left on the table, as they do not fully reap the abundant data transfer throughput inherent in DRAM datapaths. Based on the observation that the datapath of the DRAM has a hierarchical tree structure, we propose a novel, fine-grained NDP architecture for recommendation systems, which augments the DRAM datapath with an “in-DRAM” reduction unit at the DDR4/5 rank/bank-group/bank level, achieving significant performance improvements over state-of-the-art approaches. We also propose hot embedding-vector replication to alleviate the load imbalance across the reduction units.

Arithmetic modules usually have a significant impact on performance, circuit area, energy, and power in digital circuits of DSP (Digital Signal Processing). Exploring implementation trade-offs in these circuits is of utmost importance in low-power and low-cost devices such as sensors in IoT devices which often have stringent requirements. Multipliers are of particular concern due to their ubiquitous use in DSP algorithms and their inherent implementation complexity. This work proposes a framework to efficiently generalize and explore different compositions of arithmetic operators with an emphasis on parallel binary multipliers, guiding the designer through the micro-architecture development. Several partial product encoders were combined with multiple compression trees to generate multipliers that were synthesized in a commercial 65 nm to obtain area, power, and timing results.

The tunnel field-effect transistor (TFET) is a promising solution for high energy-efficient circuits. Based on the band-to-band tunneling (BTBT) condition, fast switching characteristic with a steep subthreshold swing (SS) in the ultralow-voltage operation is feasible. Our prior work has demonstrated that the SS and ON-state current can be improved without leakage current penalty through the usage of SiGe low-bandgap material in the epitaxial tunnel layer (ETL). ETL-TFET is highly compatible with the CMOS process, enabling heterogeneous integration of TFET and MOSFET in the same technology. In this work, the circuit performance of ETL-TFET and fully depleted SOI (FDSOI) MOSFET is evaluated and compared in terms of energy and delay metrics. By combining the advantages of TFET and MOSFET, heterogeneous pMOS-NTFET dynamic logic gates are proposed. The pMOS-NTFET-based logic gates demonstrate the lowest energy consumption than other realizations. Asynchronous datapath is leveraged to combat the timing variations in the ultralow-voltage region. A 20.9%–33.9% energy reduction is achieved compared with the conventional MOSFET counterpart.

This paper presents a low-power, programmable acoustic signal processor for hearing assistive devices with speech intelligibility enhancement. The reprogrammable design provides considerable flexibility for the devices to deal with personal conditions of hearing loss. A spectral-change enhancement (SCE) algorithm is implemented to improve speech intelligibility. The power consumption is minimized by adding dedicated hardware accelerators. The short-time objective intelligibility (STOI) measure is utilized for optimizing the datapath architecture. Optimization on the critical MAC operations results in 34% power and area reductions when compared to the direct-mapped design. A 50% reduction in SRAM storage is also achieved owing to the reduced memory storage for the associated MAC operations. With the aid of the optimized MAC unit and data buffer, the overall execution time is reduced by 99.2%. Designed in a 40-nm CMOS technology, the processor integrates 431k gates in area of 0.3 mm2. The design dissipates 1.5 mW at a clock frequency of 10.5 MHz from a 0.7V supply, with a processing latency of 1.05 ms.

Diverse applications of Deep convolution neural networks (CNNs), such as image classification, semantic segmentation, video recognition, etc., in smart systems require high-throughput acceleration for real-time performance. Such CNNs when realized on edge devices of the Internet of Things, a power/energy-efficient compute platform is required, which can meet the limited power/energy budget of the devices. In this regard, an end-to-end power-optimized acceleration for the compute-intensive CNNs is proposed in this work. The proposed architecture, termed WinDConv, introduces a scheme to support both regular and energy-efficient Winograd convolutions on the same architecture through a fused datapath. Furthermore, using a thoroughly investigated data sparsity enhancement, the data reuse scheme, and a suitable memory hierarchy for power efficiency, the proposed architecture is able to exhibit a practical average power efficiency of at least 12.35 tera operations per second per Watt, which is at least $2\times $ higher than the generic $z$ -first storage baseline architecture with over $3\times $ higher energy efficiency. The proposed architecture also demonstrates the applicability of the proposed schemes in commonly occurring variants of the convolution operation.

Next generation wireless communication introduces ultra high speed and low latency requirements to baseband processors, especially for multi-user multiple-input-multiple-output (MU-MIMO) systems. Polar codes, as the channel coding scheme in the control plane of 5G eMBB scenario, still cannot satisfy the hardware implementation requirements of polar decoders for MU-MIMO systems. In this brief, a high throughput and low latency polar decoder is proposed for MU-MIMO systems. The decoding architecture is fully unfolded to maximize hardware parallelism, and is fully pipelined by decoupling the round-trip scheduling datapath to enable simultaneous processing of multiple codewords. Besides, a hybrid decoding datapath is introduced by proposing complementary decoding units with mixed data representations to reduce hardware complexity. For (1024, 512) polar codes, a decoding throughput of 100.6 Gbps is achieved, and the latency is 0.43 us when 36 users are processed simultaneously.

Among its various improvements over prior NVIDIA GPUs, the NVIDIA Turing GPU boasts of four key performance enhancements to effectively counter memory load-to-use stalls. First, reduced latency on L1 hits for global memory loads helps lower average memory lookup latency. Next, the ability to dynamically configure the L1 data RAM between cacheable memory and scratchpad or shared memory, enables driver software to opportunistically maximize L1 data cache size for programs with low shared memory requirements, increasing L1 hits and reducing load-to-use stalls. Finally, the twin enhancements of doubling of vector register file capacity and the addition of a dedicated scalar or uniform register file along with a uniform datapath, ease vector register pressure and enable higher warp level parallelism, leading to better latency tolerance. We find that the above enhancements combined deliver an average speedup of 11% on modern gaming applications.

This article presents the application-aware real-time edge acceleration of CNNs (AWARE-CNNs) accelerators, which is a novel architecture design methodology for real-time execution of deep learning algorithms on IoT devices. AWARE leverages the reconfigurability of field-programmable gate arrays (FPGAs) to create application-specific architectures customized to match the inherent dataflow of targeted deep neural networks and user-specified real-time requirements. The customized datapath is combined with a customized memory path to guarantee deterministic latency-aware execution over streaming data. For results and evaluation, we have developed a Chisel-based implementation of AWARE-CNN with a full integrative framework for application-specific architecture generation and synthesis (AWARE-CNN architecture compiler). Our results demonstrate the ability to execute Tiny DarkNet and shallow MobileNet inference at 120 frames/s (FPS) and 75 FPS, using only 2.8 and 3.4 W, respectively, on a Xilinx XCZU9EG FPGA. In addition, AWARE-CNN framework’s flexibility with respect to the targeted convolutional neural networks and user constraints is validated by targeting additional design points for AlexNet (as a baseline network) and Tiny YOLOv2.

A scalable deep-learning accelerator supporting the training process is implemented for device personalization of deep convolutional neural networks (CNNs). It consists of three processor cores operating with distinct energy-efficient dataflow for different types of computation in CNN training. Unlike the previous works where they implement design techniques to exploit the same characteristics from the inference, we analyze major issues that occurred from training in a resource-constrained system to resolve the bottlenecks. A masking scheme in the propagation core reduces a massive amount of intermediate activation data storage. It eliminates frequent off-chip memory accesses for holding the generated activation data until the backward path. A disparate dataflow architecture is implemented for the weight gradient computation to enhance PE utilization while maximally reuse the input data. Furthermore, the modified weight update system enables an 8-bit fixed-point computing datapath. The processor is implemented in 65-nm CMOS technology and occupies 10.24 mm2 of the core area. It operates with the supply voltage from 0.63 to 1.0 V, and the computing engine runs in near-threshold voltage of 0.5 V. The chip consumes 40.7 mW at 50 MHz with the highest efficiency and achieves 47.4 $\mu \text{J}$ /epoch of training efficiency for the customized CNN model.

With technology scaling, subthreshold leakage has dominated the overall power consumption in a design. Input vector control is an effective technique to minimize subthreshold leakage. Low leakage input vector determination is not often possible due to large design space and simulation time. Similarly, applying an appropriate minimum leakage vector (MLV) to each Register Transfer Level (RTL) module instance in a design often results in a low leakage state with significant area overhead. In this work, we propose a top-down and bottom-up approach for propagating the input vector interval to identify low leakage input vector at primary inputs of an RTL datapath. For each module, via Monte Carlo simulation, we identify a set of MLV intervals such that maximum leakage is within (say) 10% of the lowest leakage points. As the module bit width increases, exhaustive simulation to find the low leakage vector is not feasible. Further, we need to uniformly search the entire input space to obtain as many low leakage intervals as possible. Based on empirical observations, we observe self-similarity in the subthreshold leakage distribution of adder/multiplier modules with highly regular bit-slice architectures when input space is partitioned into smaller cells. This property enables the uniform search of low leakage vectors in the entire input space where the time taken for characterization increases linearly with the module size. We further process the reduced interval set with simulated annealing to arrive at the best low-leakage vector at the primary inputs. We also propose to reduce area overhead (in some cases to 0%) by choosing Primary Input (PI) MLVs such that resultant inputs to internal nodes are also MLVs. Compared to existing work, experimental results for DSP filters simulated in 16nm technology demonstrated leakage savings of 93.6% and 89.2% for top-down and bottom-up approaches with no area overhead.

The increasing importance of inference algorithms, such as neural networks (NNs), principle component analysis (PCA), and singular value decomposition (SVD), etc., has led to the emergence of hardware accelerators to address power-performance tradeoffs in their implementation. Their large data sets make DRAM access the bottleneck for power and performance. Private SRAM scratch-pad memory is used to mitigate the DRAM access penalty but it is a limited resource in size and bandwidth. Thus, accelerator design is not just about computation, but also how data flow is scheduled across the memory hierarchy, including DRAM, scratch-pad SRAM, and datapath registers. Current accelerator design tools automate the generation of customized datapaths to improve performance, but have limited support for reducing DRAM/SRAM accesses during the computation. In this paper, we propose a memory-driven accelerator design methodology for large-scale inference applications, to maximize data access in the datapath and SRAM. We demonstrate its efficacy using several key kernels from large-scale inference applications.

Small-footprint implementations of the advanced encryption standard (AES) algorithm are of interest in resource-constrained applications like Internet of Things (IoT). Symmetries in AES allow the datapath to be scaled down to the S-Box width of 8 bits, but the ShiftRows operation leads to a potential data hazard that must be avoided. The common method for resolving the ShiftRows hazard wastes power by moving data through a sequence of pipelined registers. We present in this article a novel 8-bit AES architecture that solves data movement inefficiencies by renaming registers and saves clock power with a single state update per AES round. We then extend register renaming to include microarchitectural randomization to mitigate susceptibility to side-channel attacks, which are a concern especially for low power implementations of AES. We fabricate and evaluate our designs in a commercial 16-nm FinFET technology. Testchip measurements show that the register renaming architecture encrypts data at 0.55 pJ/bit at nominal voltage, a $2.2\times $ improvement over a state-of-the-art reference 8-bit design implemented in the same technology. Side-channel evaluation indicates that the randomized variant of register renaming significantly reduces vulnerability to differential power analysis (DPA).

Advanced Internet-of-Things applications require control-oriented codes to be executed with low latency for fast responsivity while their advanced signal processing and decision making tasks require computational capabilities. For this context, we propose three multi-issue core designs featuring an exposed datapath architecture with high performance, while retaining energy-efficiency. These features are achieved with exploitation of instruction-level parallelism, fast branching and the use of an instruction register file. With benchmarks in control-flow and signal processing application domains we measured in the best case 64% reduced energy consumption compared to a state-of-the-art RISC core, while consuming less silicon area. A high-performance design point reaches nearly 2.6 GHz operating frequency in the best case, over 2× improvement, while simultaneously achieving a 14% improvement in system energy-delay product.

The division datapath is undoubtedly the most complex operation in a wide range of digital signal processing applications, such as in adaptive filtering algorithms. This paper proposes an optimized and approximate integer divider hardware architecture, based on the Newton–Raphson algorithm combining both fixed-point dynamic range and truncation techniques, to speed up that operation. Adaptive filters have been much studied over time, as they comprise one of the most challenging fields in signal processing. This work presents dedicated hardware architectures based on normalized least mean square adaptive filtering algorithms for the power line harmonics interference cancelling. The hardware architectures are based on 2’s complement representation and were described in VHDL and synthesized into a 65 nm CMOS dedicated ASIC. Our results show that the increased approximation level of Newton–Raphson divider approximation presents up to 223 times less power dissipation than the baseline version without our optimization and approximations, providing up to 93 times of power dissipation savings in the complete interference canceller system.

In recent years, dedicated hardware accelerators for the acceleration of the convolutional neural network (CNN) have been extensively studied. Although many studies have presented efficient designs on FPGAs for image classification neural network models such as AlexNet and VGG, there are still little implementations for CNN-based object detection applications. This paper presents an OpenCL-based high-throughput FPGA accelerator for the YOLOv2 object detection algorithm on Arria-10 GX1150 FPGA. The proposed hardware architecture adopts a scalable pipeline design to support multi-resolution input image and full 8-bit fixed-point datapath to improve hardware resource utilization. Layer fusion technology that merges the convolution, batch normalization and Leaky-ReLU is also developed to avoid transmission of intermediate data between FPGA and external memory. Experimental results show that the final design achieves a peak throughput of 566 GOP/s under the working frequency of 190 MHz. The accelerator can execute YOLOv2 inference computation ( $$288\times 288$$ resolution) and tiny YOLOv2 ( $$416\times 416$$ resolution) at the speed of 35 and 71 FPS, respectively.

Since the dawn of the Internet of Things (IoT), data and system security has been the major concern for developers. Because most IoT devices operate on 8-bit controllers with limited storage and computation power, encryption and decryption need to be implemented at the transmitting and receiving ends, respectively, using lightweight ciphers. We present novel architectures for hardware implementation for the ANU cipher and present results associated with each architecture. The ANU cipher is implemented at 4-, 8-, 16-, and 32-bit datapath sizes on four different field-programmable gate array (FPGA) platforms under the same implementation condition, and the results are compared on every performance metric. Unlike previous ANU architectures, the new architectures have parallel substitution boxes (S-boxes) for high throughput and hardware optimization. With these different datapath designs, ANU cipher proves to be the obvious choice for implementing security in extremely resource-constrained systems.

Purpose The purpose of this work is to reduce the power consumption of KSA and to improve the PDP for data path applications. In digital Very Large – Scale Integration systems, the addition of two numbers is one of the essential functions. This arithmetic function is used in the modern digital signal processors and microprocessors. The operating speed of these processors depends on the computation of the arithmetic function. The speed computation block for most of the datapath elements was adders. In this paper, the Kogge–Stone adder (KSA) is designed using XOR, AND and proposed OR gates. The proposed OR gate has less power consumption due to the less number of transistors. In arithmetic logic circuit power, delay and power delay products (PDP) are considered as the important parameters. The delays reported for the proposed OR gate are less when compared with the conventional Complementary Metal Oxide Semiconductor (CMOS) OR gate and pre-existing logic styles. The proposed circuits are optimized in terms of power, delay and PDP. To analyze the performance of KSA, extensive Cadence Virtuoso simulations are used. From the simulation results based on 45 nm CMOS process, it was observed that the proposed design has obtained 688.3 nW of power consumption, 0.81 ns of delay and 0.55 fJ of PDP at 1.1 V. Design/methodology/approach In this paper, a new circuit for OR gate is proposed. The KSA is designed using XOR, AND and proposed OR gates. Findings The proposed OR gate has less power consumption due to the less number of transistors. The delays reported for the proposed OR gate are less when compared with the conventional CMOS OR gate and pre-existing logic styles. The proposed circuits are optimized in terms of power, delay and PDP. Originality/value In arithmetic logic circuit power, delay and PDP are considered as the important parameters. In this paper, a new circuit for OR gate is proposed. The power consumption of the designed KSA using the proposed OR gate is very less when compared with the conventional KSA. Simulation results show that the performance of the proposed KSA are improved and suitable for high speed applications.

Networks-on-chip (NoCs) acquired substantial advancements as the typical solution for a modular, flexible and high performance communication infrastructure coping with the scalable Multi-/Manycores technology. However, the increasing chip complexity heading towards thousand cores, together with the approaching dark-silicon era, puts energy efficiency as an integral design key for future NoC-based multicores, where NoCs are significantly contributing to the total chip power. In this paper, we propose HPPT-NoC, a dark-silicon inspired energy-efficient hierarchical TDM NoC with online distributed setup-scheme. The proposed network makes use of the dim silicon parts of the chip to hierarchically connect quad-routers units. Normal routers operate at full-chip-frequency at high supply level, and hierarchical routers operate at half-chip-frequency and lower supply voltage with adequate synchronization. Routers follow a proposed TDM architecture that separates the datapath from the control-setup planes. This allows separate clocking and operating supplies between data and control and to keep the control-setup as a single-slot-cycle design independent of the datapath slot size. The proposed NoC architecture is evaluated versus a base NoC from the state-of-the-art in terms of performance and hardware results using Synopsys VCS and Synopsys Design Compiler for SAED90nm and SAED32nm technologies. The obtained results highlight the power-frequency-trading feature supported by the proposed hierarchical NoC through the configurable data-control clock relation and maintained over the different technology nodes. With the same power budget of the base NoC, the proposed architecture provides up to 74% setup latency enhancement, 32% increased NoC saturation load, and 21% higher success rates, offering up to 78% improved power delay product. On the other hand, with 38% power savings, the proposed NoC provides up to 37% enhanced latency and 15% higher success rates, with 72% enhanced power delay product. The proposed design consumes almost double the area of the base NoC, however with an average of 56% under-clocked (dim) silicon area operating at half to quarter the maximum chip frequency. This results in reduced power density as a main concern in the dark-silicon era down to 24% of the base NoC.

In this brief, a low resource utilization field-programmable gate array (FPGA)-based long short-term memory (LSTM) network architecture for accelerating the inference phase is presented. The architecture has low-power and high-speed features that are achieved through overlapping the timing of the operations and pipelining the datapath. Moreover, this architecture requires negligible internal memory size for storing the intermediate data leading to low resource utilization and simple routing, which provides lower interconnect delay (higher operating frequency). A designer may adjust the resource utilization (as well as the latency) of the proposed architecture readily at the register-transfer level (RTL) design by adjusting the amount of parallelization. This makes the process of mapping the architecture onto different types of FPGAs, subject to defined constraints, a simple one. The efficacy of the proposed architecture is assessed by implementing an LSTM network on different types of FPGAs. Compared with the recent works, the proposed architecture provides up to about $1.6\times $ , $43.6\times $ , $21.9\times $ , and $114.5\times $ improvements in frequency, power efficiency, GOP/s, and GOP/s/W, respectively. Finally, our proposed architecture operates at 17.64 GOP/s, which is $2.31\times $ faster than the best previously reported results.

This study presents an 8-bit serialised architecture of SEED block cipher for constrained devices. The circuit utilises 356 FPGA slices and 447 1-bit registers flip-flops (FFs) in the BASYS3 board, operates with an 8-bit datapath and is aimed for use on area constraints devices. In order to keep the usage of hardware resources to a minimum but, at the same time, achieve a high level of security, the key generation process of SEED is implemented through an on-the-fly procedure. In addition, the necessary S-boxes are implemented using composite field arithmetic without using any block RAMs, resulting in a very compact implementation. The proposed architecture achieves a maximum frequency equal to 125 MHz with a total latency of 280 clock cycles and a throughput up to 57.1 Mbps for encryption or decryption.

Convolutional neural network (CNN) inference on mobile devices demands efficient hardware acceleration of low-precision (INT8) general matrix multiplication (GEMM). The systolic array (SA) is a pipelined 2D array of processing elements (PEs), with very efficient local data movement, well suited to accelerating GEMM, and widely deployed in industry. In this work, we describe two significant improvements to the traditional SA architecture, to specifically optimize for CNN inference. Firstly, we generalize the traditional scalar PE, into a Tensor-PE, which gives rise to a family of new Systolic Tensor Array (STA) microarchitectures. The STA family increases intra-PE operand reuse and datapath efficiency, resulting in circuit area and power dissipation reduction of as much as 2.08x and 1.36x respectively, compared to the conventional SA at iso-throughput with INT8 operands. Secondly, we extend this design to support a novel block-sparse data format called density-bound block (DBB). This variant (STA-DBB) achieves a 3.14x and 1.97x improvement over the SA baseline at iso-throughput in area and power respectively, when processing specially-trained DBB-sparse models, while remaining fully backwards compatible with dense models., Comment: Accepted by IEEE Computer Architecture Letters on 3/4/2020

International audience; Modern embedded computing platforms used within Cyber-Physical Systems (CPS) are nowadays leveraging more and more often on heterogeneous computing substrates, such as newest Field Programmable Gate Array (FPGA) devices. Compared to general purpose platforms, which have a fixed datapath, FPGAs provide designers the possibility of customizing part of the computing infrastructure, to better shape the execution on the application needs/features, and offer high efficiency in terms of timing and power performance, while naturally featuring parallelism. In the context of FPGA-based CPSs, this article has a two fold mission. On the one hand, it presents an analysis of the Damped Least Square (DLS) algorithm for a perspective hardware implementation. On the other hand, it describes the implementation of a robotic arm controller based on the DLS to numerically solve Inverse Kinematics problems over a heterogeneous FPGA. Assessments involve a Trossen Robotics WidowX robotic arm controlled by a Digilent ZedBoard provided with a Xilinx Zynq FPGA that computes the Inverse Kinematic.

In the Bitcoin network, computing double SHA-256 values consumes most of the network energy. Therefore, reducing the power consumption and increasing the processing rate for the double SHA-256 algorithm is currently an important research trend. In this paper, we propose a high-data-rate low-power hardware architecture named the compact message expander (CME) double SHA-256. The CME double SHA-256 architecture combines resource sharing and fully unrolled datapath technologies to achieve both a high data rate and low power consumption. Notably, the CME algorithm utilizes the double SHA-256 input data characteristics to further reduce the hardware cost and power consumption. A review of the literature shows that the CME algorithm eliminates at least 9.68% of the 32-bit XOR gates, 16.49% of the 32-bit adders, and 16.79% of the registers required to calculate double SHA-256. We synthesized and laid out the CME double SHA-256 using CMOS $0.18~\mu m$ technology. The hardware cost of the synthesized circuit is approximately 13.88% less than that of the conventional approach. The chip layout size is $5.9 mm \times 5.9 mm$ , and the correctness of the circuit was verified on a real hardware platform (ZCU 102). The throughput of the proposed architecture is 61.44 Gbps on an ASIC with Rohm 180nm CMOS standard cell library and 340 Gbps on a FinFET FPGA 16nm Zynq UltraScale+ MPSoC ZCU102.

This paper describes HighwayNoC, a Network-on-chip (NoC) that approaches ideal network performance using a Dual Data Rate (DDR) datapath. Based on the observation that routers datapath is faster than control, a DDR NoC allows flits to be routed at DDR improving throughput to rates defined solely by the datapath, rather than by the control. DDR NoCs can use pipeline bypassing to reduce packet latency at low traffic load. However, existing DDR routers offer bypassing only on in-network, non-turning hops to simplify the required logic. HighwayNoC extends bypassing support of DDR routers to local ports, allowing flits to enter and exit the network faster. Moreover, it simplifies the DDR switch allocation and the interface of router ports reducing power and area costs. Post place and route results in 28 nm technology show that HighwayNoC performs better than current state of the art NoCs. Compared to previous DDR NoCs, HighwayNoC reduces average packet latency by 7.3-27% and power consumption by 1-10%, without affecting throughput. Compared to existing Single Data Rate NoCs, HighwayNoC achieves 17-22% higher throughput, has similar or up to 13.8% lower packet latency, and mixed energy efficiency results.

Internet of Things (IoT) applications have massive client connections to cloud servers, and the number of networked IoT devices is remarkably increasing. IoT services require both low-tail latency and high concurrency in datacenters. This study aims to determine whether an order of magnitude improvement is possible in tail latency and concurrency in mainstream systems by proposing a hardware–software codesigned labeled network stack (LNS) for future datacenters. The key innovation is a cross-layered payload labeling mechanism that distinguishes different requests by payload across the full network stack, including application, TCP/IP, and Ethernet layers. This type of design enables prioritized data packet processing and forwarding along the full datapath, such that latency-insensitive requests cannot significantly interfere with high-priority requests. We build a prototype datacenter server to evaluate the LNS design against a commercial X86 server and the mTCP research, using a cloud-supported IoT application scenario. Experimental results show that the LNS design can provide an order of magnitude improvement in tail latency and concurrency. A single datacenter server node can support over 2 million concurrent long-living connections for IoT devices as a 99-percentile tail latency of 50 ms is maintained. In addition, the hardware–software codesign approach remarkably reduces the labeling and prioritization overhead and constrains the interference of high-priority requests to low-priority requests.

This letter presents an error-detection-and-correction-free voltage scaling technique based on spatial–temporal error spreading and voltage dithering (ESVD) for video processing datapaths. By leveraging the persistence of vision and the tendency of the human eye to mix pixels in close proximity, the proposed ESVD alleviates the reduction in visual quality caused by timing errors due to the lower supply voltage without explicit detection and correction; therefore, the proposed EVSD requires neither architectural nor circuit-level modification of the datapath and is especially suitable for fixed-latency video processing datapaths. Using a framebuffer-less frame-locked video scaling engine as a test case, the test chip implemented in a 40-nm 1.1-V CMOS process demonstrates that the datapath’s power consumption can be reduced maximally by 35% when scaling video streams in real time (22.6% if the efficiency of the on-chip regulator is considered). Additionally, the measured qualities in terms of the PSNR are larger than 30 dB compared with the error-free frame, whereas the maximum quality of experience from the user’s perspective in terms of the mean opinion score is 5 based on the degradation category rating (DCR).

The rapid evolution of field-programmable gate array (FPGA) devices has strongly influenced both the design methodology and development tools. This article describes an original implementation based on a hardware structure used for static and dynamic task scheduling. The proposed custom processor has hardware-implemented RTOS (HW-RTOS) features and is verified using an FPGA circuit. The solution replaces the classical stack save concept with a resource remapping mechanism that enables a new task to be executed starting with the next processor cycle. The proposed hardware scheduler enables unified management of events and interrupts, by implementing a method of attaching interrupts to tasks while ensuring the requirements of real-time systems. The robustness and performance of the proposed platform are guaranteed by the context switch operations, presence of the intertask synchronization and communication mechanisms, and by the efficient use of the multiplexed resources. Instead of saving and restoring general-purpose registers into the memory, the latency is removed by directly commuting the datapath task resources.

As AI-based applications become pervasive, CPU vendors are starting to incorporate matrix engines within the datapath to boost efficiency. Systolic arrays have been the premier architectural choice as matrix engines in offload accelerators. However, we demonstrate that incorporating them inside CPUs can introduce under-utilization and stalls due to limited register storage to amortize the fill and drain times of the array. To address this, we propose RASA, Register-Aware Systolic Array. We develop techniques to divide an execution stage into several sub-stages and overlap instructions to hide overheads and run them concurrently. RASA-based designs improve performance significantly with negligible area and power overhead., This paper is accepted to DAC 2021

High-resolution studies of data center traffic at the network core uncover short-term bursty traffic patterns, periods of high buffer utilization that last for tens of microseconds and lead to packet loss and longer flow completion time tails. While recent attention has been directed towards studying the bursty traffic at the network core, less heed has been given to the origins of bursty traffic, e.g., host machines. In this study, we try to perform high-resolution traffic measurements in the host networking stack to quantify the impact of system software components on traffic burstiness. We enforce per-packet timestamping on the datapath using various techniques like NIC timestamping, eBPF, and direct kernel source modification and measure the gaps between egress packets under various configurations. We provide preliminary findings on how process scheduling can affect traffic burstiness.

Datacenter systems and I/O devices now run at single-digit microsecond latencies, requiring ns-scale operating systems. Traditional kernel-based operating systems impose an unaffordable overhead, so recent kernel-bypass OSes [73] and libraries [23] eliminate the OS kernel from the I/O datapath. However, none of these systems offer a general-purpose datapath OS replacement that meet the needs of μs-scale systems.' AB@This paper proposes Demikernel, a flexible datapath OS and architecture designed for heterogenous kernel-bypass devices and μs-scale datacenter systems. We build two prototype Demikernel OSes and show that minimal effort is needed to port existing μs-scale systems. Once ported, Demikernel lets applications run across heterogenous kernel-bypass devices with ns-scale overheads and no code changes.

Personalized recommendation systems are gaining significant traction due to their industrial importance. An important building block of recommendation systems consists of the embedding layers, which exhibit a highly memory-intensive characteristic. A fundamental primitive of embedding layers is the embedding vector gathers followed by vector reductions, exhibiting low arithmetic intensity and becoming bottlenecked by the memory throughput. To tackle such a challenge, recent proposals employ a near-data processing (NDP) solution at the DRAM rank-level, achieving impressive performance speedups. We observe that prior rank-level-parallelism-based NDP solutions leave significant performance potential on the table as they do not fully reap the abundant transfer throughput inherent in DRAM datapaths. We propose TRiM, an NDP architecture for accelerating recommendation systems. Based on the observation that the DRAM datapath has a hierarchical tree structure, TRiM augments the DRAM datapath with “in-DRAM” reduction units at the DDR4/5 rank/bank-group/bank level. We modify the interface of DRAM to provide commands effectively to multiple reduction units running in parallel. We also propose a host-side architecture with hot embedding-vector replication to alleviate the load imbalance that arises across the reduction units. An optimal TRiM design based on DDR5 achieves up to a 7.7 × and 3.9 × speedup and reduces by 55% and 50% the energy consumption of the embedding vector gather and reduction over the baseline and the state-of-the-art NDP architecture with minimal area overhead equivalent to 2.66% of DRAM chips.

Posit arithmetic has better dynamic range and accuracy over the conventional IEEE-754 floating-point. In this paper, architectures for posit adder/subtractor and multiplier have been proposed that work as per the desired bit size of operands. Here, 16 and 32-bit architectures with different exponent size (ES) have been designed. FPGA implementations of these architectures have been done on the Xilinx Virtex-7 xc7vx330t-3ffg1157 device. In comparison with existing work, it has been observed that the usage of LUTs with ES=2 is lowered by 7.11/16.07% for 16/32-bit adders, and 1.81% for 32-bit multiplier with ES=3. In addition, 16/32-bit adder architectures require 31.47/31.66% lesser datapath delay. The 16/32-bit posit multipliers consume 10.40/10.84% lesser delay.

Simultaneous multithreading (SMT) processors target resource under-utilization related performance problems of superscalar processors by allowing various datapath resources to be simultaneously shared among several threads with ease. This approach exploits thread-level parallelism on top of the well-studied instruction level parallelism. However, when datapath resources are shared among threads, there is always a possibility that corunning ill-intended programs may try to sniff on other programs with sensitive information like passwords or cryptocurrency information. By stress-testing and measuring many of the shared resources, this is quite achievable. These approaches are known as side-channel attacks. SMT-COP, a study published recently, is a secure architectural implementation based on resource scheduling that prevents these kind of attacks based on execution logic with a modest performance loss of nearly 10%. In this study, we aim to improve SMT-COP’s resource distribution scheme with several approaches, providing a quite close sense of security to that of SMT-COP while improving the performance by up to 8.86% on the average, over the original SMT-COP design.