Search

Field-programmable gate array (FPGA) provides rich parallel computing resources with high energy efficiency, making it ideal for deep convolutional neural network (CNN) acceleration. In recent years, automatic compilers have been developed to generate network-specific FPGA accelerators. However, with more cascading deep CNN algorithms adapted by various complicated tasks, reconfiguration of FPGA devices during runtime becomes unavoidable when network-specific accelerators are employed. Such reconfiguration can be difficult for edge devices. Moreover, network-specific accelerator means regeneration of RTL code and physical implementation whenever the network is updated. This is not easy for CNN end users. In this article, we propose a domain-specific FPGA overlay processor, named OPU to accelerate CNN networks. It offers software-like programmability for CNN end users, as CNN algorithms are automatically compiled into executable codes, which are loaded and executed by OPU without reconfiguration of FPGA for switch or update of CNN networks. Our OPU instructions have complicated functions with variable runtimes but a uniform length. The granularity of instruction is optimized to provide good performance and sufficient flexibility, while reducing complexity to develop microarchitecture and compiler. Experiments show that OPU can achieve an average of 91% runtime multiplication and accumulation unit (MAC) efficiency (RME) among nine different networks. Moreover, for VGG and YOLO networks, OPU outperforms automatically compiled network-specific accelerators in the literature. In addition, OPU shows $5.35\times $ better power efficiency compared with Titan Xp. For a real-time cascaded CNN networks scenario, OPU is $2.9\times $ faster compared with edge computing GPU Jetson Tx2, which has a similar amount of computing resources.

Lightweight convolutional neural networks (LW-CNNs) such as MobileNet, ShuffleNet, SqueezeNet, etc., have emerged in the past few years for fast inference on embedded and mobile system. However, lightweight operations limit acceleration potential by GPU due to their memory bounded nature and their parallel mechanisms that are not friendly to SIMD. This calls for more specific accelerators. In this paper, we propose an FPGA-based overlay processor with a corresponding compilation flow for general LW-CNN accelerations, called Light-OPU. Software-hardware co-designed Light-OPU reformulates and decomposes lightweight operations for efficient acceleration. Moreover, our instruction architecture considers sharing of major computation engine between LW operations and conventional convolution operations. This improves the run-time resource efficiency and overall power efficiency. Finally, Light-OPU is software programmable, since loading of compiled codes and kernel weights completes switch of targeted network without FPGA reconfiguration. Our experiments on seven major LW-CNNs show that Light-OPU achieves 5.5x better latency and 3.0x higher power efficiency on average compared with edge GPU NVIDIA Jetson TX2. Furthermore, Light-OPU has 1.3x to 8.4x better power efficiency compared with previous customized FPGA accelerators. To the best of our knowledge, Light-OPU is the first in-depth study on FPGA-based general processor for LW-CNNs acceleration with high performance and power efficiency, which is evaluated using all major LW-CNNs including the newly released MobileNetV3.

Autonomous mobile robots need perception module to understand nearby environments, avoid obstacles and, most importantly, interact with the humans in the dynamic environments. In order to perform human robot interaction for mobile robots, face detection is an inevitable and important function. In recent years, deep learning has emerged to be the most accurate approach for face detection problems. However, deep learning models have very high computational density, therefore cannot be executed in real-time on the embedded low-end processors that most of the robot systems equip with. Switching to high-end CPU or GPU generally means much higher cost and power consumption. Meanwhile, FPGA is well suited for robot application because it provides high computing power with low cost and high power efficiency. In this paper we design an FPGA based system for the acceleration of MTCNN, which is one of the most accurate face detection neural networks. We evaluate the system using a Zynq 706 board and compare the performance with embedded CPU and popular GPU edge computing platform Jetson TX2. Our FPGA system has 40× lower latency than Jetson TX2 with 2.5 × higher power efficiency, making it a promising candidate for egde computing on mobile robot applications.

This paper presents the overview of an overlay architecture on FPGA (OPU). It is applicable to general CNN acceleration with software like programmablility and fast compilation time. Good portability is guaranteed as only executable codes are reloaded for application-switch without FPGA re-synthesis. Our OPU instructions have complicated functions with variable runtimes but have a uniform length of 32 bits. This large granularity of instructions makes it easier to develop compiler and micro-architecture. Thus, OPU is a good candidate for domain specific processor without large volume. Experiments results show that OPU can achieve around 90 percent of runtime computing resource utilization (PE efficiency) among eight different network applications, which is 2% – 48% better compared with existing customized accelerators.

Because layout stage consumes the lion share of FPGA synthesis runtime, pre-layout power estimation can be viewed as an early stage estimation and is needed for power minimization at the early design stage. Consisting two phases of feature selection and model training, data mining is effective for data based modeling, yet it has not been applied in a rigid fashion for FPGA power estimation as the existing algorithms can be viewed as model training using features selected manually. In this paper, we apply machine learning with automatic feature selection to pre- and post- logic synthesis estimations, named pre-synthesis and post-synthesis estimation. Experiments using Lattice Diamond MachXO2 family show that compared to the post-layout power simulation, post-synthesis estimation is 20x faster with 8.62% average error, while pre-synthesis estimation is 600x faster with considerably larger error that still needs further improvement. Furthermore, compared to existing algorithms using manually selected features, our post-synthesis estimation using automatic feature selection reduces error by 2-3 times. Finally, the ranking of features is able to provide insights for power minimization.