Search

In prior parts of this series, I analyzed the 1) number of all issued patents and issued computer architecture patents; 2) the prosecution time and effective patent term; 3) the number of claims, breakdown of independent and dependent claims, and effect that excess claim fees had on the numbers of total and independent claims; and 4) the type of claims (apparatus, method, or Beauregard), and effect that the Supreme Court's decision in Alice v. CLS Bank had on the number of independent and dependent method claims, for patents that were issued to 18 leading computer architecture companies and filed between 1996 and 2020. This article examines the 1) number of "backward" citations to previously issued/published U.S. patents and U.S. patent publications, foreign patents, and Other References and 2) the number of "forward" citations to a company's patent by another U.S. patent or U.S. patent publication. [ABSTRACT FROM AUTHOR]

Neural networks (NNs) are growing in importance and complexity. An NN's performance (and energy efficiency) can be bound either by computation or memory resources. The processing-in-memory (PIM) paradigm, where computation is placed near or within memory arrays, is a viable solution to accelerate memory-bound NNs. However, PIM architectures vary in form, where different PIM approaches lead to different tradeoffs. Our goal is to analyze, discuss, and contrast dynamic random-access memory (DRAM)-based PIM architectures for NN performance and energy efficiency. To do so, we analyze three state-of-the-art PIM architectures: 1) UPMEM, which integrates processors and DRAM arrays into a single 2-D chip, 2) Mensa, a 3-D-stacking-based PIM architecture tailored for edge devices, and 3) SIMDRAM, which uses the analog principles of DRAM to execute bit-serial operations. Our analysis reveals that PIM greatly benefits memory-bound NNs: 1) UPMEM provides 23× the performance of a high-end graphics processing unit (GPU) when the GPU requires memory oversubscription for a general matrix–vector multiplication kernel, 2) Mensa improves energy efficiency and throughput by 3.0× and 3.1× over the baseline Edge tensor processing unit for 24 Google edge NN models, and 3) SIMDRAM outperforms a central processing unit/graphics processing unit by 16.7×/1.4× for three binary NNs. We conclude that the ideal PIM architecture for NN models depends on a model's distinct attributes, due to the inherent architectural design choices. [ABSTRACT FROM AUTHOR]

In this article, we present a hardware architecture optimized for sparse and dense matrix processing in TensorFlow Lite and compatible with embedded-heterogeneous devices that integrate central processing unit and field-programmable gate array (FPGA) resources. The fused architecture for dense and sparse matrices design offers multiple configuration options that tradeoff parallelism and complexity, and uses a dataflow model to create four stages that read, compute, scale, and write results. All stages are designed to support TensorFlow Lite operations including asymmetric quantized activations, column-major matrix write, per-filter/per-axis bias values, and current scaling specifications. The configurable accelerator is integrated with the TensorFlow Lite inference engine running on the ARMv8 processor. We compare performance/power/energy with the state-of-the-art RUY software multiplication library showing up to 18× acceleration and 48× in dense and sparse modes, respectively. The sparse mode benefits from structural pruning to fully utilize the digital signal processing blocks present in the FPGA device. [ABSTRACT FROM AUTHOR]

Always-on TinyML perception tasks in Internet of Things applications require very high energy efficiency. Analog compute-in-memory (CiM) using nonvolatile memory (NVM) promises high energy efficiency and self-contained on-chip model storage. However, analog CiM introduces new practical challenges, including conductance drift, read/write noise, fixed analog-to-digital (ADC) converter gain, etc. These must be addressed to achieve models that can be deployed on analog CiM with acceptable accuracy loss. This article describes AnalogNets: TinyML models for the popular always-on tasks of keyword spotting (KWS) and visual wake word (VWW). The model architectures are specifically designed for analog CiM, and we detail a comprehensive training methodology, to retain accuracy in the face of analog nonidealities, and low-precision data converters at inference time. We also describe AON-CiM, a programmable, minimal-area phase-change memory (PCM) analog CiM accelerator, with a layer-serial approach to remove the cost of complex interconnects associated with a fully pipelined design. We evaluate the AnalogNets on a calibrated simulator, as well as real hardware, and find that accuracy degradation is limited to 0.8%/1.2% after 24 h of PCM drift (8 bits) for KWS/VWW. AnalogNets running on the 14-nm AON-CiM accelerator demonstrate 8.55/26.55/56.67 and 4.34/12.64/25.2 TOPS/W for KWS and VWWs with 8-/6-/4-bit activations, respectively. [ABSTRACT FROM AUTHOR]

Analyzes the historical patenting behavior and patent characteristics of computer architecture companies. [ABSTRACT FROM AUTHOR]

This two-part article takes a break from my series on the Analysis of Historical Patenting Behavior and Patent Characteristics of Computer Architecture Companies in order to analyze the patents that were issued to 18 computer architecture companies in 2021. Part III of the series will return in the July/August issue of IEEE Micro. This two-part article first examines the number of total patents and the number of computer architecture patents that were issued to each company in 2021. Second, it compares the distribution of those patents across different patent classes. Third, it highlights one patent from each company that may be particularly notable. [ABSTRACT FROM AUTHOR]

The supercomputer "Fugaku" is an exascale manycore-based parallel system developed as a Japanese national flagship supercomputer in the FLAGSHIP 2020 Project. While "Fugaku" was ranked first for several benchmarks such as TOP500, HPCG, HPL-AI, and Graph500 in 2020, the major design concept is the application-first concept by the co-design for power efficiency and high performance. We have designed an original manycore processor based on Armv8 instruction sets with the scalable vector extension, A64FX processor, with Fujitsu, our industry partner. The system consists of 158,976 nodes with 7.6 million cores in total and a theoretical peak of 537 Peta Floating-point Operations Per Second in double-precision floating points, connected by Tofu-D interconnect. The high performance computing (HPC)-oriented design enables an extremely good performance for memory-intensive workloads thanks to HBM2 memory with breakthrough power efficiency. In this article, we present the pragmatic practice of our co-design effort for "Fugaku" followed by an overview and the performance of "Fugaku." [ABSTRACT FROM AUTHOR]

The effective implementation of quantization depends not only on the specific task but also on the hardware resources. This article presents a hardware-aware customized quantization method for convolutional neural networks. We propose a learnable parameter soft clipping full integer quantization (LSFQ), which includes weight and activation quantization with the learnable clipping parameters. Moreover, the LSFQ accelerator architecture is customized on the field-programmable gate array (FPGA) platform to verify the hardware awareness of our method, in which DSP48E2 is designed to realize the parallel computation of six low-bit integer multiplications. The results showed that the accuracy loss of LSFQ is less than 1% compared with the full-precision models including VGG7, mobile-net v2 in CIFAR10, and CIFAR100. An LSFQ accelerator was demonstrated at the 57th IEEE/ACM Design Automation Conference System Design Contest (DAC-SDC) and won the championship at the FPGA track. [ABSTRACT FROM AUTHOR]

In this article, an energy-efficient deep learning processor is proposed for deep neural network (DNN) training in mobile platforms. Conventional mobile DNN training processors suffer from high-bit precision requirement and high ReLU-dependencies. The proposed processor breaks through these fundamental issues by adopting three new features. It first combines the runtime automatic bit precision searching method addition to both conventional dynamic fixed-point representation and stochastic rounding to realize low-precision training. It adopts bit-slice scalable core architecture with the input skipping functionality to exploit bit-slice-level fine-grained sparsity. The iterative channel reordering unit helps the processor to maintain high core utilization by solving the workload unbalancing problem during zero-slice skipping. It finally achieves at least 4.4× higher energy efficiency compared with the conventional DNN training processors. [ABSTRACT FROM AUTHOR]

The authors propose a heterogeneous floating-point (FP) computing architecture to maximize energy efficiency by separately optimizing exponent processing and mantissa processing. The proposed exponent-computing-in-memory architecture and mantissa-free exponent-computing algorithm reduce the power consumption of both memory and FP MAC while resolving previous FP computing-in-memory processors' limitations. Also, a bfloat16 DNN training processor with proposed features and sparsity exploitation support is implemented and fabricated in 28-nm CMOS technology. It achieves 13.7-TFLOPS/W energy efficiency while supporting FP operations with CIM architecture. [ABSTRACT FROM AUTHOR]

The papers in this special section explore cutting edge research on topics that combine artificial intelligence with edge computing, relating to the design, performance, or application of microprocessors and microcomputers. [ABSTRACT FROM AUTHOR]

In their 2018 Turing Award lecture and 2019 paper, John Hennessy and David Patterson reviewed computer architecture progress since the 1960s. They projected a second golden age akin to the first, approximately 1986–1996, when new instruction set architectures, almost all reduced instruction set computers (RISCs), revolutionized the industry, eliminated most minicomputer vendors, rivaled mainframes, and began a takeover of supercomputing. The C language and derivatives came to pervade systems programming, whereas Unix derivatives came to run many servers, desktops, and smartphones. Such outcomes were not inevitable but depended on evolutionary interactions of computer architecture and languages, industry dynamics, and sometimes random coincidences. [ABSTRACT FROM AUTHOR]

With advances in device technologies, design methodologies, and programming paradigms, microprocessors of today have undergone a complete metamorphosis compared to their predecessors from the 1970s and 1980s. In particular, heterogeneity has become an all-pervasive feature of modern-day processors in device and packaging technologies, architectures, and programming interfaces, resulting in unprecedented enhancements in their performance, power efficiency, and functionality. We explore these heterogeneous designs, from their inception in the early 2000s, to their culmination into a whole new class of processors termed as Systems-on-a-Chip, and finally track their evolution into the microprocessors of tomorrow. [ABSTRACT FROM AUTHOR]

Reports on the history, development, and applications supported by Intel's microprocessor product line. Microprocessors have evolved substantially since the introduction of the Intel 4004 half a century ago. Over the years, several processors have stood out for their innovative designs, commercial successes, forays into new application domains, and significant performance and efficiency enhancements compared to their competitors. We polled the computer architecture community, and received votes from 254 processor designers and engineers, researchers from academia and industry, faculty, and graduate students, identifying their favorite processors from every decade since the 1970s, as well as their favorite processor of all time. The results of the poll are presented in this article. [ABSTRACT FROM AUTHOR]

Reports on the technology of microprocessors, focusing on the history and development of the DEC Alpha 21164. [ABSTRACT FROM AUTHOR]

It was a different computing world in the late 1980s. Many if not most researchers in the computer architecture area had become convinced that complex instruction sets such as the Intel x86 were doomed in light of the many advantages promised by reduced instruction set architecture publications. There were many voices within Intel urging upper management to abandon x86 and get started on some alternative. Even engineers who had worked on Intel's then-flagship 486 were expressing serious reservations about whether the x86 architecture could be "dragged further up the hill" to be, if not directly competitive with emerging RISC designs, at least close enough for x86 to remain profitable. [ABSTRACT FROM AUTHOR]

Reports on the history and development of computer aided design systems. In quick succession between 1964 and 1971, our field saw the proposal of Moore's law,1 the coining of the term "computer architecture,"2 and the introduction of the first microprocessor.3 For much of the five decades since then, we have benefitted extraordinarily from both the dynamism of Moore's law transistor scaling and the stable durability of the hardware–software abstractions of computer architecture. The dynamic duo of Moore's law and computer architecture have allowed massive scaling to occur, and also to be navigated smoothly with relatively little software impact. For example, in the late 1980s and early 1990s, surges in power density occurred as we reached challenging limits in very large scale integration (VLSI) designs based on bipolar transistors; a technology transition from bipolar to complementary metal–oxide–semiconductor (CMOS) occurred with relatively little impact or awareness from the software portion of the computing community.4 Over the past 10–15 years however, more fundamental shifts have occurred. For example, Dennard scaling,5 a companion phenomenon to Moore's law stating that power density could remain stable while transistor sizes shrank, is reaching physical limits. This means that further Moore's law increases in transistor counts are becoming more complex and are only achieved with great effort and at higher power-density costs. Furthermore, as we reach fundamental physical limits in the functioning of small semiconductor transistors, Moore's law itself is being challenged by the increased physical effort and financial expense required to maintain transistor scaling trends. [ABSTRACT FROM AUTHOR]

Despite its tremendous potential, it is still unclear how quantum computing will scale to satisfy the requirements of its most powerful applications. Among other issues, there are hard limits to the number of qubits that can be integrated into a single chip. Multicore architectures are a firm candidate for unlocking the scalability of quantum processors. Nonetheless, the vulnerability and complexity of quantum communications make this a challenging approach. A comprehensive design should imply consolidating the communications stack in the quantum computer architecture. In this article, we explain how this vision, by entangling communications and computation in the core of the design, may help to solve the open challenges. We also summarize the first results of our application of structured design methodologies backing this vision. With our work, we hope to contribute with design guidelines that may help unleash the potential of quantum computing. [ABSTRACT FROM AUTHOR]

Codesign has been an integral part of computer architecture since the very first systems were brought online. From the early days of the field until now, end-user applications inevitably shape the design and capabilities of subsequent generations of hardware. Likewise, the characteristics and capabilities of new computational hardware and systems often impact the algorithms and software that run on them. Quantum computing (QC) is similarly reliant on codesign approaches, particularly now in its resource constrained early days. This article discusses what codesign means in a QC setting, gives examples of its value to QC, and proposes key attributes of QC codesign approaches going forward. [ABSTRACT FROM AUTHOR]

Tremendous success of machine learning (ML) and the unabated growth in model complexity motivated many ML-specific designs in hardware architectures to speed up the model inference. While these architectures are diverse, highly optimized low-precision arithmetic is a component shared by most. Nevertheless, recommender systems important to Facebook's personalization services are demanding and complex: They must serve billions of users per month responsively with low latency while maintaining high prediction accuracy. Do these low-precision architectures work well with our production recommendation systems? They do. But not without significant effort. In this article, we share our search strategies to adapt reference recommendation models to low-precision hardware, our optimization of low-precision compute kernels, and the tool chain to maintain our models' accuracy throughout their lifespan. We believe our lessons from the trenches can promote better codesign between hardware architecture and software engineering, and advance the state of the art of ML in industry. [ABSTRACT FROM AUTHOR]

Low-precision techniques can effectively reduce the computational complexity and bandwidth requirements of a convolutional neural network (CNN) inference, but may lead to significant accuracy degradation. Mixed-low-precision techniques provide a superior approach for CNN inference since it can take the advantages of low precision while maintaining accuracy. In this article, we propose a high-performance, highly flexible ${W^8A^8}$W8A8 (INT8 weight and INT8 activation) and ${W^T A^2}$WTA2 (TERNARY weight and INT2 activation) mixed-precision CNN inference hardware architecture, DPUmxp, designed and implemented on Xilinx Virtex UltraScale+13P FPGA with peak performance up to 58.9 TOPS. [ABSTRACT FROM AUTHOR]

The open-source hardware community contributes a variety of processors and accelerators, but combining them effectively into a complete System-on-Chip (SoC) remains a difficult task. We present a design flow for the seamless hardware and software integration of accelerators into a complete SoC and for its evaluation through rapid FPGA-based prototyping. By leveraging ESP, our open-source platform for agile heterogeneous SoC design, we demonstrate FPGA prototypes of various SoC designs, featuring the NVIDIA Deep Learning Accelerator and the Ariane RISC-V 64-bit processor core. [ABSTRACT FROM AUTHOR]

The articles in this special section focus on compiling for accelerators. These articles are divided into two groups according to how the accelerator is coupled with the microprocessor. The first set of articles describes novel compilation techniques targeting accelerators that are CPU ISA extensions, such as vectorization and matrix multiplication. In the second half, advanced compilation techniques that use dedicated and FPGA-based accelerators are discussed. [ABSTRACT FROM AUTHOR]

Every year, IEEE Micro selects some of the most notable articles published in the previous year's computer architecture conferences to highlight in this Special Issue. In early 2021, the selection committee chose 12 articles to appear as "Top Picks" and an additional 12 deserving an "Honorable Mention" from among the 2020 conferences. [ABSTRACT FROM AUTHOR]

Cloud computing has begun a transformation from using virtual machines to using containers. Containers are attractive because of their "build once, run anywhere" computing model and their minimal performance overhead. Cloud providers leverage the lean nature of containers to run hundreds of them or more on a few cores. Furthermore, containers enable the serverless paradigm, which involves the creation of short-lived processes. In this work, we identify that containerized environments create page translations that are extensively replicated across containers in the TLB and in page tables. The result is high TLB pressure and redundant kernel work during page table management. To remedy this situation, this article proposes BabelFish, a novel architecture to share page translations across containers in the TLB and in page tables. BabelFish reduces the mean and tail latency of containerized workloads, cold-start effects of function execution, and container bring-up time. This work also advocates for the need to provide more hardware support for containerized and serverless environments. [ABSTRACT FROM AUTHOR]

Fault-tolerant quantum computing is required to execute many of the most promising quantum applications. In recent years, numerous error correcting codes, such as the surface code, have emerged which are well suited for current and future limited connectivity 2-D devices. We find quantum memory, particularly resonant cavities with transmon qubits arranged in a 2.5-D architecture, can efficiently implement surface codes with around 20× fewer transmons via this work. We virtualize 2-D memory addresses by storing the code in layers of qubit memories connected to each transmon. Distributing logical qubits across many memories has minimal impact on fault tolerance and results in substantially more efficient logical operations. Virtualized logical qubit (VLQ) systems can achieve fault tolerance comparable to conventional 2-D transmon-only architectures while putting within reach a proof-of-concept experimental demonstration of around ten logical qubits, requiring only 11 transmons and 9 attached cavities. [ABSTRACT FROM AUTHOR]

Deep learning has sparked a renaissance in computer systems and architecture. Despite the breakneck pace of innovation, there is a crucial issue concerning the research and industry communities at large: how to enable neutral and useful performance assessment for machine learning (ML) software frameworks, ML hardware accelerators, and ML systems comprising both the software stack and the hardware. The ML field needs systematic methods for evaluating performance that represents real-world use cases and useful for making comparisons across different software and hardware implementations. MLPerf answers the call. MLPerf is an ML benchmark standard driven by academia and industry (70+ organizations). Built out of the expertise of multiple organizations, MLPerf establishes a standard benchmark suite with proper metrics and benchmarking methodologies to level the playing field for ML system performance measurement of different ML inference hardware, software, and services. [ABSTRACT FROM AUTHOR]

The superconductor single-flux-quantum (SFQ) logic family has been recognized as a promising solution for the post-Moore era, thanks to the ultrafast and low-power switching characteristics of superconductor devices. Researchers have made tremendous efforts in various aspects, especially in device and circuit design. However, there has been little progress in designing a convincing SFQ-based architectural unit due to a lack of understanding about its potentials and limitations at the architectural level. This article provides the design principles for SFQ-based architectural units with an extremely high-performance neural processing unit (NPU). To achieve our goal, we developed and validated a simulation framework to identify critical architectural bottlenecks in designing a performance-effective SFQ-based NPU. We propose SuperNPU, which outperforms a conventional state-of-the-art NPU by 23 times in terms of computing performance and 1.23 times in power efficiency even with the cooling cost of the 4K environment. [ABSTRACT FROM AUTHOR]

We are building HALO, a flexible ultralow-power processing architecture for implantable brain– computer interfaces (BCIs) that directly communicate with biological neurons in real time. This article discusses the rigid power, performance, and flexibility tradeoffs that BCI designers must balance, and how we overcome them via HALO's palette of domain-specific hardware accelerators, general-purpose microcontroller, and configurable interconnect. Our evaluations using neuronal data collected in vivo from a nonhuman primate, along with full-stack algorithm to chip codesign, show that HALO achieves flexibility and superior performance per watt versus existing implantable BCIs. [ABSTRACT FROM AUTHOR]

General matrix multiplication (GEMM) is pervasive in various domains, such as signal processing, computer vision, and machine learning. Conventional binary architectures for GEMM exhibit poor scalability in area and energy efficiency, due to the spatial nature of number representation and computing. On the contrary, unary computing processes data in temporal domain with extremely simple logic. However, to date, there rarely exist efficient architectures for unary GEMM. In this work, we first present uGEMM, a hardware-efficient unary GEMM architecture enabled by universally compatible arithmetic units, which simultaneously achieves input-insensitivity and high output accuracy. Next, we demonstrate that the proposed uGEMM can reliably early terminate the computation and offers dynamic energy-accuracy scaling for real-world applications via an accuracy-aware metric. Finally, to propel the future research for unary computing, we open source our unary computing simulator, UnarySim. [ABSTRACT FROM AUTHOR]

The rapidly growing compute demands of AI necessitate the creation of new computing architectures and approaches. Tenstorrent designed its architecture (embodied in Grayskull and Wormhole devices) to tackle this challenge via two fundamental and synergistic approaches. The first is via compute-on-packets fabric that is built from ground up for massive scaleout. The second is the ability to execute dynamic computation, built into the compiler, runtime software and hardware architecture. By combining these approaches, TensTorrent will enable continued scaling of AI workloads. [ABSTRACT FROM AUTHOR]

Data-parallel problems demand ever growing floating-point (FP) operations per second under tight area- and energy-efficiency constraints. In this work, we present Manticore, a general-purpose, ultraefficient chiplet-based architecture for data-parallel FP workloads. We have manufactured a prototype of the chiplet’s computational core in Globalfoundries 22FDX process and demonstrate more than 5x improvement in energy efficiency on FP intensive workloads compared to CPUs and GPUs. The compute capability at high energy and area efficiency is provided in “Snitch: A tiny pseudo dual-issue processor for area and energy efficient execution of floating-point intensive workloads,” IEEE Trans. Comput., containing eight small integer cores, each controlling a large floating-point unit (FPU). The core supports two custom ISA extensions: The SSRs extension elides explicit load and store instructions by encoding them as register reads and writes (“Stream semantic registers: A lightweight RISC-V ISA extension achieving full compute utilization in single-issue cores,” IEEE Trans. Comput.). The floating-point repetition extension decouples the integer core from the FPU allowing floating-point instructions to be issued independently. These two extensions allow the single-issue core to minimize its instruction fetch bandwidth and saturate the instruction bandwidth of the FPU, achieving FPU utilization above 90%, with more than 40% of core area dedicated to the FPU. [ABSTRACT FROM AUTHOR]

Chiplet technology can significantly reduce the cost and time needed to develop custom high-performance silicon products. To realize a chiplet-based product, a die-to-die (D2D) network to interconnect the chiplets is required. Almost all of today's chiplet-based products use proprietary D2D interfaces. Industry has primarily paid attention to D2D PHYs. A design also requires logical information flow between chiplets. Current designs use proprietary PHYs and logic protocols between components. This approach makes it challenging to integrate chiplets from multiple vendors. The open domain-specific architecture (ODSA) is a project within the open compute project (OCP) community that aims to establish open physical and logical D2D interfaces for chiplets. Ultimately, the ODSA aims to create open interfaces to enable a marketplace of interoperable chiplets. This will allow product designers to develop chiplet-based products that integrate best in class chiplets from multiple vendors. This article reviews technical developments within the ODSA. [ABSTRACT FROM AUTHOR]

PCI Express (PCIe) specification has been doubling the data rate every generation in a backward compatible manner every two to three years. PCIe 6.0 specification will adopt PAM-4 signaling at 64.0 GT/s for maintaining the same channel reach of prior generations. A forward error correction (FEC) mechanism will offset the high BER of PAM-4. We propose a new flit-based approach with a lightweight, low-latency FEC coupled with a strong cyclic redundancy check (CRC) and a low-latency link-level retry mechanism to meet the stringent low-latency, high-bandwidth, and high-reliability goals. We also present a new low-power state that ensures power consumption is proportional to bandwidth usage without impacting the traffic flow. [ABSTRACT FROM AUTHOR]

With increasingly complex neural network architectures and heterogeneous device characteristics, finding a reasonable graph partitioning and device placement strategy is challenging. There have been prior attempts at learned approaches for solving device placement, these approaches are computationally expensive, unable to handle large graphs consisting over 50000 nodes, and do not generalize well to unseen graphs. To address all these limitations, we propose an efficient single-shot, generalized deep RL method (SGDP) based on a scalable sequential attention mechanism over a graph neural network that is transferable to new graphs. On a diverse set of representative deep learning models, our method on average achieves 20% improvement over human placement and 18% improvement over the prior art with 15× faster convergence. We are the first to demonstrate super human performance on 8-layer recurrent neural network language model and 8-layer GNMT consisting of over 50000 nodes, on 8-GPUs. We provide rationales and sensitivity study on model architecture selections. [ABSTRACT FROM AUTHOR]

Neural architecture search (NAS) finds favorable network topologies for better task performance. Existing hardware-aware NAS techniques only target to reduce inference latency on single CPU/GPU systems and the searched model can hardly be parallelized. To address this issue, we propose ColocNAS, the first synchronization-aware, end-to-end NAS framework that automates the design of parallelizable neural networks for multidevice systems while maintaining a high task accuracy. ColocNAS defines a new search space with elaborated connectivity to reduce device communication and synchronization. ColocNAS consists of three phases: 1) offline latency profiling that constructs a lookup table of inference latency of various networks for online runtime approximation; 2) differentiable latency-aware NAS that simultaneously minimizes inference latency and task error; and 3) reinforcement-learning-based device placement fine-tuning to further reduce the latency of the deployed model. Extensive evaluation corroborates ColocNAS's effectiveness to reduce inference latency while preserving task accuracy. [ABSTRACT FROM AUTHOR]

For five years, OpenPiton has provided hardware designs, build and verification scripts, and other infrastructure to enable efficient, detailed research into manycores and systems-on-chip. It enables open-source hardware development through its open design and support of a plethora of open simulators and CAD tools. OpenPiton was first designed to perform cutting-edge computer architecture research at Princeton University and opening it up to the public has led to thousands of downloads and numerous academic publications spanning many subfields within computing. In this article, we share some of the lessons learned during the development of OpenPiton, provide examples of how OpenPiton has been used to efficiently test novel research ideas, and discuss how OpenPiton has evolved due to its open development and feedback from the open-source community. [ABSTRACT FROM AUTHOR]

Demanded by ever-evolving data processing algorithms, field-programmable gate arrays (FPGAs) have become essential components of modern computing systems, thanks to their reconfigurable and distributed computing capabilities. However, FPGAs are among the very few integrated chips that still require long development cycles and high human efforts, even for industrial vendors. In this article, we introduce OpenFPGA, an open-source framework that can automate and significantly accelerate the development cycle of customizable FPGA architectures. OpenFPGA allows users to customize their FPGA architectures down to circuit-level details using a high-level architecture description language and autogenerate associated Verilog netlists which can be used in a backend flow to generate production-ready layouts. A generic Verilog-to-Bitstream generator is also provided, allowing end-users to implement practical applications on any FPGAs that OpenFPGA can support. Using OpenFPGA, we demonstrate less than 24-h layout generation of two FPGA fabrics, which are based on a Stratix-like architecture built with a commercial 12-nm standard cell library and 40-nm custom cells, respectively. [ABSTRACT FROM AUTHOR]

Programmable hardware accelerators (e.g., vector processors, GPUs) have been extremely successful at targeting algorithms with regular control and memory patterns to achieve order-of-magnitude performance and energy efficiency improvements. However, they perform far under the peak on important irregular algorithms, like those from graph processing, database querying, genomics, advanced machine learning, and others. This work posits that the primary culprit is specific forms of irregular control flow and memory access. By capturing the problematic behavior at a domain-agnostic level, we propose an accelerator that is sufficiently general, matches domain-specific accelerator performance, and significantly outperforms traditional CPUs and GPUs. [ABSTRACT FROM AUTHOR]

Varifocal storage (VS) presents a new architecture that coordinates application demands, hardware accelerators, and intelligent data storage devices to efficiently support various input resolutions of system components, but still maintain the flexibility and quality without additional costs. Instead of faithfully shipping the raw data, the cross-layer design of VS allows an intelligent storage device to work directly with the running application to generate and deliver data sets in the desired resolution and quality before going through the narrower system interconnect. In this way, VS minimizes the bandwidth demand from the data source and allows hardware accelerators to work on received data without additional preprocessing. Without programmers' hints, VS achieves 1.46× speedup on a computer with approximate hardware accelerators. [ABSTRACT FROM AUTHOR]

The growing computational requirements of AI applications are challenging today's general-purpose CPU and GPU architectures and driving the need for purpose-built, programmable AI solutions. Habana Labs designed its Goya processor to meet the high throughput/low latency demands of Inference workloads, and its Gaudi processor for throughput combined with massive scale up and scale out capability needed to speed training workloads efficiently. To address the need for scaling training, Habana is the first AI chip developer to integrate standard Ethernet onto a training processor. [ABSTRACT FROM AUTHOR]

Recent years have seen an explosion of demand for high-performance, high-efficiency compute available at scale. This demand has skyrocketed with the move to the public cloud and 5G networking, where compute nodes must operate within strict latency constraints and power budgets. The Neoverse N1 platform is Arm's latest high end offering from a scalable portfolio of IP for high performance and energy efficient machines. [ABSTRACT FROM AUTHOR]

This article presents the recent advancements on the Advanced IO Processor (AIOP), a network processor architecture designed by NXP Semiconductors. The AIOP is a multicore accelerated computing architecture where each core is equipped with dedicated hardware for rapid task switching on every hardware accelerator call. A hardware preemption controller snoops on the accelerator completions and sends task preemption requests to the cores, thus reducing the latency of real-time tasks. A technique of priority thresholding is used to avoid latency uncertainty on lower priority tasks and head-of-line blocking. In this way, the AIOP handles the conflicting requirements of high throughput and low latency for next-generation wireless applications such as WiFi and 5G. In presence of frequent preemptions, the throughput reduces by only 3% on AIOP, compared to 25% on a similar network processor. Moreover, the absolute throughput and latency numbers are 2X better. The area and power overhead of adding hardware task-scheduling and preemption is only about 3%. [ABSTRACT FROM AUTHOR]

HPC architects are currently facing myriad challenges from ever tighter power constraints and changing workload characteristics. In this article, we discuss the current state of FPGAs within HPC systems. Recent technological advances show that they are well placed for penetration into the HPC market. However, there are still a number of research problems to overcome; we address the requirements for system architectures and interconnects to enable their proper exploitation, highlighting the necessity of allowing FPGAs to act as full-fledged peers within a distributed system rather than attached to the CPU. We argue that this model requires a reliable, connectionless, hardware-offloaded transport supporting a global memory space. Our results show how our fully fledged hardware implementation gives latency improvements of up to 25% versus a software-based transport, and demonstrates that our solution can outperform the state of the art in HPC workloads such as matrix–matrix multiplication achieving a 10% higher computing throughput. [ABSTRACT FROM AUTHOR]