Search

Convolutional neural networks (CNNs) have been widely used in many tasks, but training CNNs is time consuming and energy hungry. Using the low-bit integer format has been proved promising for speeding up and improving the energy efficiency of CNN inference, while CNN training can hardly benefit from such a technique because of the following challenges: 1) the integer data format cannot meet the requirements of the data dynamic range in training, resulting in the accuracy drop; 2) the floating-point data format keeps sizeable dynamic range with much more exponent bits, thus using it results in higher accumulation power than using the integer data format; and 3) there are some specially designed data formats (e.g., with group-wise scaling) that have the potential to deal with the former two problems but common hardware platforms cannot support them efficiently. To tackle all these challenges and make the training phase of CNNs benefit from the low-bit format, we propose a low-bit training framework for CNNs to pursue a better tradeoff between accuracy and energy efficiency: 1) we adopt element-wise scaling to increase the dynamic range of data representation, which significantly reduces the quantization error; 2) group-wise scaling with hardware friendly factor format is designed to reduce the element-wise exponent bits without degrading the accuracy; and 3) we design the customized hardware unit that implements the low-bit tensor convolution arithmetic with our multilevel scaling data format. Experiments show that our framework achieves a superior tradeoff between the accuracy and the bit-width than previous low-bit training studies. For training various models on CIFAR-10, using 1-bit mantissa and 2-bit exponent is adequate to keep the accuracy loss within 1%. On larger datasets like ImageNet, using 4-bit mantissa and 2-bit exponent is adequate. Through the energy consumption simulation of the whole network, we can see that training a variety of models with our framework could achieve $4.9\times $ – $10.2\times $ higher energy efficiency than full-precision arithmetic. [ABSTRACT FROM AUTHOR]

Convolutional neural networks (CNNs) are widely used in computer vision and natural language processing. Field-programmable gate arrays (FPGAs) are popular accelerators for CNNs. However, if used in critical applications, the reliability of FPGA-based CNNs becomes a priority because FPGAs are prone to suffer soft errors. Traditional protection schemes, such as triple modular redundancy (TMR), introduce a large overhead, which is not acceptable in resource-limited platforms. This article proposes to use an ensemble of weak CNNs to build a robust classifier with low cost. To have a group of base CNNs with low complexity and balanced similarity and diversity, residual neural networks (ResNets) with different layers (20/32/44/56) are combined in the ensemble system to replace a single strong ResNet 110. In addition, a robust combiner is designed based on the reliability evaluation of a single ResNet. Single ResNets with different layers and different ensemble schemes are implemented on the FPGA accelerator based on Xilinx Zynq 7000 SoC. The reliability of the ensemble systems is evaluated based on a large-scale fault injection platform and compared with that of the TMR-protected ResNet 110 and ResNet 20. Experiment results show that the proposed ensembles could effectively improve the system reliability when suffering soft errors with an overhead much lower than TMR. [ABSTRACT FROM AUTHOR]

Modern low-cost electronic devices can achieve high precision for global navigation satellite systems (GNSSs) and related applications. Recently, the pseudo-range and carrier phase have been directly obtained from a smartphone to establish a professional-level surveying device. Although promising results have been obtained by linking to an external GNSS antenna, the real-time kinematic (RTK) positioning performance requires further improvement when using the embedded smartphone antenna. We first investigate the observation quality characteristics of the Xiaomi Mi 8 smartphone. The carrier-to-noise-density ratio of L5/E5a signals is below that of L1/E1 signals, and the cycle slip and loss of lock are severe, especially for L5/E5a signals. Therefore, we use an improved stochastic model and ambiguity-resolution strategies to improve the short-baseline RTK positioning accuracy. Experimental results show that the ambiguity fixing rate can reach approximately 90% in 3 h of observations when using the embedded antenna, while the GPS/Galileo/BDS single-frequency combination is more suitable for smartphones. On the other hand, convergence takes 10–30 min, and the RTK positioning accuracy can reach 1 and 2 cm along the horizontal and vertical directions, respectively, if ambiguity is resolved correctly. Moreover, we verify the feasibility of using a mass-produced smartphone for deformation monitoring. Results from a simulated dynamic deformation experiment indicate that a smartphone can recognise deformations as small as 2 cm. [ABSTRACT FROM AUTHOR]