Search

Sparse convolutional neural network (SCNN) accelerators eliminate unnecessary computations and memory access by exploiting zero-valued activation pixels and filter weights. However, data movement between the multiplier array and accumulator buffer tends to be a performance bottleneck. Specifically, the scatter network, which is the core block of SCNN accelerators, delivers Cartesian products to the accumulator buffer, and certain products are not immediately delivered owing to bus contention. A previous SCNN-based architecture eliminates bus contention and improves the performance significantly by making use of different dataflows. However, it relies only on weight sparsity, and its performance is highly dependent on the workload. In this paper, we propose a novel scatter network architecture for SCNN accelerators. First, we propose network topologies (such as window and split queuing), which define the connection between the FIFOs and crossbar buses in the scatter network. Second, we investigate arbitration algorithms (such as fixed priority, round-robin, and longest-queue-first), which define the priorities of the products delivered to the accumulator buffer. However, the optimization of the scatter network architecture alone may not be able to provide sufficient performance gain since it does not help to reduce bus contention itself. In this paper, we propose a cubic-constrained bank allocation for the accumulator buffer, which reduces bus contention without any increase in the hardware area. Based on the results of cycle-accurate simulation, register-transfer-level (RTL) design, and logic synthesis, this study investigates the trade-off between the performance and complexity of SCNN accelerators. In detail, it is verified that, when the optimized SCNN accelerators are applied to AlexNet, the proposed scatter network architecture can remove most of the performance degradation due to bus contention, thereby improving the accelerator performance by 72%, with an area increase of 18%. It is also shown that the proposed bank allocation provides an additional performance gain of up to 31% when it is applied to SqueezeNet. The proposed scatter network architectures and bank allocation can eliminate bus contention in most Cartesian product-based accelerators, regardless of the workload, without changing accelerator components other than the scatter network.

The increasing significance of technology in daily lives led to the need for the development of convenient methods of human-computer interaction (HCI). Given that the existing HCI approaches exhibit various limitations, hand gesture recognition-based HCI may serve as a more intuitive mode of human-machine interaction in many situations. In addition, the system has to be deployable on low-power devices for applicability in broadly defined Internet of Things (IoT) and smart home solutions. Recent advances exhibit the potential of deep learning models for gesture classification, whereas they are still limited to high-performance hardware. Embedded neural network accelerators are constrained in terms of available memory, central processing unit (CPU) clock speed, graphics processing unit (GPU) performance, and a number of supported operations. The aforementioned problems are addressed in this paper by namely two approaches - simplifying the signal processing pipeline to avoid recurrent structures and efficient topological design. This paper employs an intuitive scheme allowing for the generation of the data in the compressed form from the sequence of range-Doppler images (RDI). Thus, it allows for the design of a neural classifier avoiding the usage of recurrent layers. The proposed framework has been optimized for Intel® Neural Compute Stick 2 (Intel® NCS 2), at the same time achieving promising classification accuracy of 97.57%. To confirm the robustness of the proposed algorithm, five independent persons have been involved in the algorithm testing process.

Polynomial multiplications based on the number theoretic transform (NTT) are critical in lattice-based post-quantum cryptography algorithms. Therefore, this paper presents a platform-agnostic unified hardware accelerator design (Unif-NTT) to compute the forward and inverse operations of the NTT for the CRYSTALS-Kyber algorithm. Moreover, a unified design (Unif-BU) of the Cooley-Tukey and Gentleman-Sande butterflies is presented using two adders, multipliers, subtractors, routing multiplexers and barret-based modular reduction units. Finally, a dedicated controller is implemented for efficient control functionalities. The implementation results are realized on field-programmable gate array (FPGA) and application-specific integrated circuit (ASIC) platforms. The Unif-NTT requires 1664 and 1792 clock cycles for one forward and inverse NTT computations, respectively. It can operate up to a maximum frequency of $212MHz$ and $2.5GHz$ over Virtex-7 FPGA and 28nm ASIC platforms, respectively. The Unif-NTT is 26% more efficient in Area-Time-Product compared to the most area-optimized NTT accelerator from the state-of-the-art. The Unif-NTT design is suited for applications that demand reasonable hardware resources with processing speed.

We propose in this paper an efficient FALCON accelerator called EFX based on a HW/SW co-design where FALCON is a post-quantum cryptographic (PQC) scheme tailored as a digital signature algorithm (DSA). Our findings reveal that FALCON exhibits unique characteristics and structures which distinguish it from other PQC-DSAs. A key finding is that, unlike its counterparts, FALCON doesn’t prioritize a single, time-consuming task; instead, it processes a variety of tasks with comparable execution times. Consequently, the conventional methods focusing on accelerating dominant few tasks, which are generally effective for other algorithms, prove less efficient for FALCON, especially concerning the minimization of the silicon area used. To overcome this, we strategically focus on the granular optimization of lower-level operations rather than on broader functional segments, aiming to boost performance while conserving hardware space. Moreover, to mitigate the potential degradation due to limitation of hardware resources, we have implemented a pipelined execution strategy for the FALCON functions and refined the sampling function–a critical task that is challenging to accelerate due to inherent sequential algorithm–enabling it to run concurrently on both software and hardware, thus reducing latency. Our hardware design, synthesized at $300MHz$ using Samsung’s $28nm$ and $45nm$ process technologies, demonstrates superior performance in generating FALCON signatures, with a $3.58 \times $ improvement in clock cycles over an existing hardware accelerator. EFX occupies 38K $um ^{2}$ and 74K $um ^{2}$ for $28nm$ and $45nm$ processes, respectively, comparatively small compared to other PQC accelerators.

Heterogeneous computing relies on collaboration among different types of processors on shared data. In systems with discrete accelerators (e.g., GP-GPU), data sharing requires transferring a large amount of data between CPU and accelerator memories and can significantly increase the end-to-end execution time. This paper proposes a novel mechanism called Demand MemCpy (DMC) to hide the data sharing overheads. DMC copies data from host memory to accelerator memory based on demands at page granularity. It utilizes a hardware-only mechanism to fetch the requested page with a short latency and the background pre-copy to fetch related pages in advance. Our evaluation shows that DMC can reduce the end-to-end execution time of GP-GPU application by 25.4% on average by overlapping computation with data transfer and not transferring unused pages.

Traffic sign recognition (TSR) technology allows the vehicle to recognize road signs through a camera and use it for driving. For traffic safety, TSR is one of the core technologies constituting advanced driver assistance systems (ADAS), and several researches have been studied. The advent of convolutional neural networks (CNNs) has opened up new possibilities in automotive environments, especially for ADAS. However, deploying a real-time TSR application in resource-constrained ADAS is challenging because most CNNs require high computing resources and memory usage. To address this problem, some works have been studied to consider optimization in embedded platforms, but existing works used many hardware resources or showed low computation performance. In this paper, we propose a low-cost CNN-based real-time TSR hardware accelerator. Firstly, we extend a novel hardware-friendly quantization method to reduce computational complexity. The quantization method can reconstruct the CNN so that all operations, including the skip connection path of residual blocks, use only integer arithmetic and reduce the computational overhead by replacing the quantization affine mapping process with a shift operation. Secondly, the proposed hardware accelerator applied two parallelization strategies to balance real-time inference and resource consumption. In addition, we present a simple and effective hardware design scheme that handles the skip connection path of residual blocks. This design scheme can optimize the dataflow of the skip connection path and reduce additional internal memory usage. Experimental results show that the reconstructed fully integer-based CNN only requires 24M integer operations (IOPs) and possesses a model size of 0.17MB. Compared with the previous work, the proposed CNN model size was reduced by $\times 105$ , and the number of operations was reduced by $\times 58$ . In addition, the proposed CNN can achieve a TSR accuracy of 99.07%, which is the highest accuracy among CNN-based TSR works implemented on embedded platforms. The proposed hardware accelerator achieves a computation performance of 960 MOPS and a frame rate of 40 FPS when implemented on a Xilinx ZC706 SoC. Consequently, this work improves by $\times 11.87$ and $\times 36.7$ on computation performance and frame rate compared to the previous work.

The secure hash algorithm 2 (SHA-2) family, including the SHA-224/256/384/512 hash functions, is widely adopted in many modern domains, ranging from Internet of Things devices to cryptocurrency. SHA-2 functions are often implemented on hardware to optimize performance and power. In addition to the high-performance and low-cost requirements, the hardware for SHA-2 must be highly flexible for many applications. This paper proposes an SHA-2 hardware architecture named the multimode SHA-2 accelerator (MSA), which has high performance and flexibility at the system-on-chip level. To achieve high performance and flexibility, our accelerator applies three optimal techniques. First, a multimode processing element architecture is proposed to enable the accelerator to compute various SHA-2 functions for many applications. Second, a three-stage arithmetic logic unit pipeline architecture is proposed to reduce the critical paths and hardware resources. Finally, nonce generator and nonce validator architectures are proposed to reduce memory access and maximize the performance of the proposed MSA for blockchain mining applications. The MSA accuracy is tested on a real hardware platform (the Xilinx Alveo U280 FPGA). The experimental results on the field programmable gate array (FPGA) prove that the proposed MSA achieves significantly better performance, hardware efficiency, and flexibility than previous works. The evaluation results for energy efficiency show that the proposed MSA achieves up to 38.05 Mhps/W, which is 543.6 and 29 times better than the state-of-the-art Intel i9-10940X CPU and RTX 3090 GPU, respectively.

Nowadays, the use of hardware accelerators to boost the performance of HPC applications is a consolidated practice, and among others, GPUs are by far the most widespread. More recently, some data centers have successfully deployed also FPGA accelerated systems, especially to boost machine learning inference algorithms. Given the growing use of machine learning methods in various computational fields, and the increasing interest towards reconfigurable architectures, we may expect that in the near future FPGA based accelerators will be more common in HPC systems, and that they could be exploited also to accelerate general purpose HPC workloads. In view of this, tools able to benchmark FPGAs in the context of HPC are necessary for code developers to estimate the performance of applications, as well as for computer architects to model that of systems at scale. To fulfill these needs, we have developed FER (FPGA Empirical Roofline), a benchmarking tool able to empirically measure the computing performance of FPGA based accelerators, as well as the bandwidth of their on-chip and off-chip memories. FER measurements enable to draw Roofline plots for FPGAs, allowing for performance comparisons with other processors, such as CPUs and GPUs, and to estimate at the same time the performance upper-bounds that applications could achieve on a target device. In this paper we describe the theoretical model on which FER relies, its implementation details, and the results measured on Xilinx Alveo accelerator cards.

Convolutional Neural Networks (CNNs) have been shown to be very useful in image recognition and other Artificial Intelligence (AI) applications, however, at the expense of intensive computation requirement. To address the challenge of overwhelming calculation requirements, researchers have proposed various network pruning techniques. But, due to the irregular sparse patterns, unstructured sparse networks are difficult to compute efficiently on either Graphic processing units (GPUs) or Field Programmable Gate Arrays (FPGAs). In this paper, we propose a software/hardware co-optimized Reconfigurable Sparse convolutional Neural Network accelerator design (RSNN) on FPGAs. A novel sparse convolution dataflow is proposed with simpler control logic than existing mux-based selection logic. To balance the computation load on different Processing Units (PUs), we propose a software-based loadbalance aware pruning technique as well as a kernel merging method. Experimental results show that RSNN is 2.41×-7.91× better on Digital Signal Processor (DSP) efficiency than previous dense CNN FPGA accelerators, and 1.23×-2.93× better than previous sparse CNN FPGA accelerators.

Recently, many researches have been conducted on recognition of facial emotion using convolutional neural networks (CNNs), which show excellent performance in computer vision. To obtain a high classification accuracy, a CNN architecture with many parameters and high computational complexity is required. However, this is not suitable for embedded systems where hardware resources are limited. In this paper, we present a lightweight CNN architecture optimized for embedded systems. The proposed CNN architecture has a small memory footprint and low computational complexity. Furthermore, a novel hardware-friendly quantization method that uses only integer-arithmetic is proposed. The proposed hardware-friendly quantization method maps the scale factors to power-of-two terms and replaces multiplication and division operations using scale factors with shift operations. To improve the generalization and classification performance of the CNN, we create the FERPlus-A dataset. This is a new training dataset created using a variety of image processing algorithms. After training with FERPlus-A, quantization has been performed. The size of a quantized CNN parameter is about 0.39 MB, and the number of operations is about 28 M integer operations (IOPs). By evaluating the performance of the quantized CNN that uses only integer-arithmetic on the FERPlus test dataset, the classification accuracy is approximately 86.58%. It achieved higher accuracy than other lightweight CNNs in prior studies. The proposed CNN architecture that uses only integer-arithmetic is implemented on the Xilinx ZC706 SoC platform for real-time facial emotion recognition by applying parallelism strategies and efficient data caching strategies. The FPGA-based CNN accelerator implemented for real-time facial emotion recognition achieves about 10 frame per second (FPS) at 250 MHz and consumes 2.3 W.

This paper presents a comprehensive hardware accelerator architecture of YOLOv3-Tiny targeted for low-cost FPGA with a high frame rate, high accuracy, and low latency. The proposed accelerator implements all YOLO layers in hardware including zero pad layer, convolution layer, leaky ReLU layer, batch normalization layer, max-pooling layer, and up-sampling layer. The architecture is built based on data flow and control flow hybrid architecture. The data preparation and computation process work asynchronously using the data flow paradigm, while the overall governing process is controlled by proposed custom instruction set which adopts the principle of control flow paradigm. The principle of General Matrix Multiplication (GEMM) is adopted to compute the convolution process. We designed a GEMM processor using an optimum size of systolic array architecture. The systolic core is small and the overall architecture supports the multicore system, making it scalable to be implemented on larger size FPGAs. We also proposed a hardware architecture for mapping feature maps into matrix form for GEMM convolution which can save on-chip memory space. Lastly, we modified the original YOLO algorithm to further optimize it in our hardware. The modification includes reducing the bit precision to reduce memory space and bandwidth requirement, merging the normalization layer with the convolution layer to reduce arithmetic complexity, and adding a new DLQ layer to keep the bit size small while maintaining the accuracy. Based on the experimental results, our proposed design manages to achieve a frame rate of 8.3 FPS with the throughput of 31.5 GOPS, outperforming the same convolution computation that is performed by Ryzen 5 3600 CPU up to $69.3\times $ in latency. Moreover, our proposed design also has the smallest clock cycle ratio up to $1.75\times $ than other commercial accelerators. The system is useful and suitable for edge computing applications.

The Open Radio Access Network (RAN) concept has been gaining wide acceptance in recent network architectures, including 5G New Radio (NR) network deployments. Current Open RAN radio implementation efforts, aim at integrating several white-box hardware elements and executing digital processing on open-source software. When building such a software-based, 5G Open RAN platform, challenges include achieving real-time execution times for demanding computational blocks of the 5G NR physical layer processing, such as Low Density Parity Check (LDPC) decoding. In this context, having already identified both the capabilities as well as the challenges that Field-programmable Gate Arrays (FPGAs) offer for accelerating LDPC, we present our novel LDPC FPGA accelerator system. In this paper, we contribute the implementation details of our FPGA accelerator design as well as the process of integrating the accelerator with OpenAirInterface (OAI), the basis for our 5G NR platform. For the first time in the literature, we show an FPGA-based LDPC accelerator fully integrated with a complete software platform, that is able to achieve more than $1.6Gbps$ decoding throughput and up to 13 times faster execution times compared to single core software implementations. Finally, in our results, we show that LDPC encoding is more challenging to accelerate due to lower computational complexity.

Convolutional Neural Networks (CNNs) revolutionized computer vision and reached the state-of-the-art performance for image processing, object recognition, and video classification. Even though CNN inference is notoriously compute-intensive, as convolutions account for >90% of the total operation tasks, the ability to tradeoff between accuracy, performance, power, and latency to meet target application makes it an open research topic. This paper proposes the Spatial Locality Input Data (SLID) method for computational reuse during the inference stage for a pre-trained network. The method exploits input data spatial locality via skipping partial processing of the multiply-and-accumulate (MAC) operations for adjacent data and equating its value to previously computed ones. SLID improves the throughput of resource-constrained devices (Internet-of-Things, edge devices) and accelerates computations during the inference phase by reducing the number of MAC operations. Such approximate computing schema does not require a similarity quantification step nor any modification for the training stage. The computational data reuse was evaluated on three well-known distinctive CNN structures and data sets with alternating layer selections: LeNet, CIFAR-10, and AlexNet. The computational data reuse method saves up to 34.9%, 49.84%, and 31.5% of MAC operations while reducing the accuracy by 8%, 3.7%, and 5.0% for the three models mentioned earlier, respectively. Besides, the proposed method saves on memory access by eliminating data fetching of skipped inputs. Furthermore, filter size, strides, and padding on the accuracy and savings of operations are analyzed. SLID is the first work to exploit the input spatial locality for savings on CNN convolution operations with minimal accuracy loss and without memory or computational overhead. This makes it a great option to support intelligence at the edge.

In this paper, we propose a real-time hardware naive Bayes classifier (NBC) which is implemented on field programmable gate array (FPGA). We first use logarithm transformation based look-up table and float-to-fixed point process to simplify the calculations in naive Bayes classification algorithm. The methods clear up the multiplication and division operations of floating points completely. Based the simplified algorithm, we design our hardware architecture which includes both training and inference part. A novel format of logarithm look-up table with very limited items and a shifter in it are working together to calculate the logarithm value of any number. There are several processing element (PE) arrays in the accelerator where each PE in an array is running in parallel, which speed up the classification process remarkably. The experiments prove that the proposed accelerator has much better real-time efficiency than the general processor, some hardware Bayes classifiers and convolutional neural network (CNN) accelerators. It outperforms the NBC and semi-NBC accelerators and costs far less resources on chip than many CNN accelerators. Its utilization of LUT, FF and BRAM is only 10%, 0.05% and 2% of CNN accelerators on average. The experimental results over five datasets of different magnitudes show the accelerator has almost no loss of classification accuracy comparing with ARM Cortex-A9 processor. Their deviation of the classification accuracy is only 0.39% on average. What's more, it improves the performance of the training phase and the inference phase about 7.9+e4 and 8.3+e4 on average, respectively.

This paper presents a multiply-accumulate (MAC) unit that enables a dual-mode truncation error compensation (TEC) scheme based on a fixed-width Booth multiplier (FWBM) for convolutional neural network (CNN) inference operations. The proposed tailored TEC schemes of Modes 1 and 2 can achieve high MAC accuracy for a general or rectified linear unit-based CNN model with general (Mode 1) or positive/zero (Mode 2) input patterns. By pre-calculating the pre-known CNN model coefficients, the proposed dual-mode TEC scheme can be realized using minimal partial product operations with high hardware efficiency using a software-hardware codesign approach. Further, a reconfigurable architecture of the resultant MAC unit is presented to realize the proposed dual-mode TEC scheme. By evaluating the accuracy for 9-N and 25-N MAC operations (N denotes the number of times MAC is performed), a MAC operation using the proposed TEC scheme can achieve the highest accuracy for Modes 1 and 2, relative to contrast samples that directly employ the FWBM with a conventional TEC function. The hardware performances of 9-N and 25-N MAC units are also evaluated using the TSMC 40-nm standard cell library. Compared with the contrast TEC-enabled designs, the proposed MAC unit exhibits higher hardware efficiency in terms of area, delay, and power consumption and achieves a minimum reduction of more than 40% in both area-delay-error and power-delay-error products. Moreover, the resultant 9-N and 25-N MAC units are verified using a system-on-chip field-programmable gate array platform to test a CNN model for handwritten digit classification.

To effectively compute convolutional layers, a complex design space must exist (e.g., the dataflow techniques associated with the layer parameters, loop transformation techniques, and hardware parameters). For efficient design space exploration (DSE) of various dataflow techniques, namely, the weight-stationary (WS), output-stationary (OS), row-stationary (RS), and no local reuse (NLR) techniques, the processing element (PE) structure and computational pattern of each dataflow technique are analyzed. Various performance metrics are calculated, namely, the throughput (in giga-operations per second, GOPS), computation-to-communication ratio (CCR), on-chip memory usage, and off-chip memory bandwidth, as closed-form expressions of the layer and hardware parameters. In addition, loop interchange and loop unrolling techniques with a double-buffer architecture are assumed. Many roofline model-based simulations are performed to explore relevant dataflow techniques for a wide variety of convolutional layers of typical neural networks. Through simulation, this paper provides insights into the trends in accelerator performance as the layer parameters change. For convolutional layers with large input and output feature map (ifmap and ofmap) widths and heights, the GOPS of the NLR dataflow technique tends to be higher than that of the techniques. For convolutional layers with low weight and ofmap widths and heights, the RS dataflow technique achieves optimal GOPS and on-chip memory usage. In the case of convolutional layers with small weight widths and heights, the GOPS of the WS dataflow technique tends to be high. In the case of convolutional layers with small ofmap widths and heights, the OS dataflow technique achieves optimal GOPS and on-chip memory usage.

The convolutional neural network (CNN) has been widely applied in computer vision applications. Due to the intensive computation, the general central processing unit (CPU) processors are not efficient to meet the real-time requirement. Various hardware accelerators based on the application specific integrated circuit (ASIC) and field programmable gate array (FPGA) have been designed to accelerate CNN models during the inference phase. The data flow architecture has been extensively adopted because of the high parallelism. However, given the continual development in the computer vision field, CNN models have become increasingly diverse. Thus, CNN accelerators based on data flow architectures face an emerging challenge to maintain high throughput while coping with various CNN models. In this paper, we design a software-defined architecture to solve this. The goal of this study is to make the hardware change as the application changes to achieve high flexibility and high performance. In our proposed architecture, all the parts can be software-defined to cope with different CNN models. A flexible software-defined processing element (PE) array is designed to compute different weight filter sizes. In addition, a software-defined data reuse technique based on two ideal reuse cases is proposed to ensure that all the parameters need to be loaded only once during the computing phase. To support this reuse technique, we also propose the software-defined on-chip buffer so that the weight and image buffers share one dynamic buffer. By using the sparsity property of the input feature map, the full-connected (FC) layer is accelerated. About 88% of the FC weight parameters can be skipped when loading the VGG-16 model. Finally, we implemented this software-defined accelerator on the FPGA. Compared to the other FPGA based accelerators, our proposed accelerator can preserve high performance while maintaining flexibility.

Deep convolutional neural networks (DCNNs) have become one of the most popular approaches to many visual processing tasks. The majority of existing works on the accelerating DCNNs focus on high performance while neglecting the hardware resource utilization, like on-chip memory and DSP. In this paper, we propose a resources-efficient and configurable DCNN accelerator. A four-level processing-element (PE)-array-based structure is presented to realize high parallelism calculation of convolutional operation, and a new storage pattern named hybrid stationary (HS) is proposed to take full advantage of the used on-chip memory footprint and limited off-chip memory bandwidth. Moreover, roofline model is adopted to explore the design space of the given hardware resources. The proposed architecture achieves 113 G-ops/s at 100 MHz and consumes 784 DSP48 modules and 211.5 Block RAM modules on ZYNQ-7 ZC706 evaluation board. To the best of our knowledge, the proposed accelerator is the only implemented system on FGPA platform that can achieve multiple advantages: high-performanced, configurable, and efficient in power and resources utilization. It shows significant utilization improvement compared with the other available architectures.

This paper presents a low-area and energy-efficient hardware accelerator for the convolutional neural networks (CNNs). Based on the multiply-accumulate-based architecture, three design techniques are proposed to reduce the hardware cost of the convolutional computations. First, to reduce the computational cost of convolutions, an adaptive bit-width reduction combined with near-zero skipping is proposed based on differential input method (DIM). The DIM-based design technique can reduce 62.5% of operation bit-width and improve 17.0% of activation sparsity with almost ignorable CNN accuracy degradation. Second, it has been found that adopting a bi-directional filtering window in a CNN accelerator can considerably reduce the energy for data movement with a much smaller number of memory accesses. To expedite the bi-directional filtering operations, we also propose a bi-directional first-input-first-output (bi-FIFO). With SRAM bit-cell layout manner, the proposed bi-FIFO facilitates fast data re-distribution with area and energy efficiency. To verify the effectiveness of the proposed techniques, the AlexNet accelerator has been designed. The numerical results show that the proposed adaptive bit-width reduction scheme achieves 34.6% and 58.2% of area and energy savings, respectively. The bi-FIFO-based accelerator also achieves 32.8% improved processing time.