Search

Autonomous robots and their applications are becoming popular in several different fields, including tasks where robots closely interact with humans. Therefore, the reliability of computation must be paramount. In this work, we measure the reliability of Google’s Coral Edge tensor processing unit (TPU) executing three deep reinforcement learning (DRL) models through an accelerated neutrons beam. We experimentally collect data that, when scaled to the natural neutron flux, account for more than 5 million years. Based on our extensive evaluation, we quantify and qualify the radiation-induced corruption on the correctness of DRL. Crucially, our data show that the Edge TPU executing DRL has an error rate that is up to 18 times higher the limit imposed by international reliability standards. We found that despite the feedback and intrinsic redundancy of DRL, the propagation of the fault induces the model to fail in the vast majority of cases or the model manages to finish but reports wrong metrics (i.e., speed, final position, and reward). We provide insights on how radiation corrupts the model, on how the fault propagates in the computation, and about the failure characteristic of the controlled robot.

Dedicated hardware is required to efficiently execute the highly resource-demanding modern artificial neural networks (ANNs). The high complexity of ANN systems has motivated the use of high-level synthesis (HLS) tools, which increase design abstraction. Higher abstraction reduces the implementation of field-programmable gate array (FPGA) hardware details visible to the designer, making an accurate reliability evaluation challenging. When ANN hardware accelerators are used in safety-critical systems, reliability becomes paramount, and to have a realistic reliability evaluation, physical fault injection, such as beam testing, is mandatory. Existing reliability analysis approaches focus on specific ANN hardware accelerator designs, but when HLS tools are used, the tool flow and design decisions can impact reliability. Therefore, we evaluate the error rate of ANN hardware accelerators generated by HLS tools under high-energy neutrons and explore the impact of HLS parameters on reliability. Our results show that by tweaking hardware parameters, such as the reuse of resources, it can increase the error rate linearly. Furthermore, the generated ANN hardware accelerator with the best tradeoff of area and execution cycles can deliver $15\times $ more correct executions than the least optimized one, despite its increased error rate.

Several applications in which autonomous capabilities are required, such as unmanned probes used for space exploration, require high reliability but impose strict power budget limits. To enable autonomy, convolutional neural networks (CNNs) are heavily employed to detect objects in images or frames. Recently, to adapt the computing power to the application needs, the market made available several low-power and low-cost commercial-off-the-shelf (COTS) computing solutions, called EdgeAI accelerators, that are very attractive for executing neural networks with limited power requirements. Since autonomous vehicles may operate on a wide range of temperatures, it is fundamental to understand the dependence of EdgeAI device error rate on the temperature. In this work, we consider Google’s Coral Edge Tensor Processing Unit (TPU) and measure its atmospheric neutrons reliability at different temperatures, which goes from −40 °C to +90 °C. We show a decrease in the failures in time (FIT) rate of almost $4\times $ as temperature increases. Moreover, we compare the thermal neutrons and heavy ion cross sections of the TPU at room temperature. We report a difference of up to $\sim 31\times $ between atmospheric and thermal neutrons cross section on the TPU. The TPU’s cross section for heavy ions is $\sim 20\times $ higher than for atmospheric neutrons and $\sim 187\times $ than thermal neutrons.

Machine learning is among the greatest advancements in computer science and engineering and is today used to classify or detect objects, a key feature in autonomous vehicles. Since neural networks are heavily used in safety-critical applications, such as automotive and aerospace, their reliability must be paramount. However, the reliability evaluation of neural network systems is extremely challenging due to the complexity of the software, which is composed of hundreds of layers, and the underlying hardware, typically a parallel device or an embedded accelerator. This article reviews fundamental concepts of artificial intelligence, deep neural networks, and parallel computing device reliability. Then, the reliability studies that consider the radiation effects in the hardware, their propagation through the computing architecture, and their final impact on the software output are summarized. A detailed survey of the available strategies to measure the sensitivity of neural network frameworks and observe fault propagation is given, together with a summary of the data obtained so far. Finally, a discussion on how to use the experimental evaluation to design effective and efficient hardening solutions for artificial neural networks is provided. The available hardening solutions are critically reviewed, highlighting their benefits and drawbacks.

The reliability evaluation of deep neural networks (DNNs) executed on graphic processing units (GPUs) is a challenging problem, since the hardware architecture is highly complex, and the software frameworks are composed of many layers of abstraction. While software-level fault injection is a common and fast way to evaluate the reliability of complex applications, it may produce unrealistic results, since it has limited access to the hardware resources, and the adopted fault models may be too naive (i.e., single and double bit flips). Contrarily, physical fault injection with neutron beam provides realistic error rates but lacks fault propagation visibility. This article proposes a characterization of the DNN fault model combining both neutron beam experiments and fault injection at the software level. We exposed GPUs running general matrix multiplication (GEMM) and DNNs to beam neutrons to measure their error rate. On DNNs, we observe that the percentage of critical errors can be up to 61% and show that the error correction code (ECC) is ineffective in reducing critical errors. We then performed a complementary software-level fault injection, using fault models derived from register-transfer level (RTL) simulations. Our results show that by injecting complex fault models, the version 3 of you only look once (YOLOv3) misdetection rate is validated to be very close to the rate measured with beam experiments, which is $8.66\times $ higher than the one measured with fault injection using only single-bit flips.

Quantum computing (QC), by exploiting the quantum properties of quantum bits (qubits), significantly improves the performance and efficiency of computation. Unfortunately, quantum devices are very susceptible to external perturbation, including natural radiation. In this article, we measure, through GEANT4 simulations, the charge deposited by impinging neutrons in superconducting quantum devices. As we show, most atmospheric neutrons deposit sufficient energy to break the Cooper pairs and, thus, can potentially modify the qubit state. Then, with a dedicated circuit-level fault injector, we track the transient fault propagation in real quantum circuits and measure its impact on the output probabilities distribution. We discuss the fault model for qubits and show how to measure faults’ impact on the circuit output.