Search

Reduced precision computation is a key enabling factor for energy-efficient acceleration of deep learning (DL) applications. This article presents a 7-nm four-core mixed-precision artificial intelligence (AI) chip that supports four compute precisions--FP16, Hybrid-FP8 (HFP8), INT4, and INT2--to support diverse application demands for training and inference. The chip leverages cutting-edge algorithmic advances to demonstrate leading-edge power efficiency for 8-bit floating-point (FP8) training and INT4 inference without model accuracy degradation. A new HFP8 format combined with separation of the floating- and fixed-point pipelines and aggressive circuit/architecture optimization enables performance improvements while maintaining high compute utilization. A high-bandwidth ring protocol enables efficient data communication, while power management using workload-aware clock throttling maximizes performance within a given power budget. The AI chip demonstrates 3.58-TFLOPS/W peak energy efficiency and 26.2-TFLOPS peak performance for HFP8 iso-accuracy training, and 16.9-TOPS/W peak energy efficiency and 104.9-TOPS peak performance for INT4 iso-accuracy inference.

Advances in deep neural networks (DNNs) and the availability of massive real-world data have enabled superhuman levels of accuracy on many AI tasks and ushered the explosive growth of AI workloads across the spectrum of computing devices. However, their superior accuracy comes at a high computational cost, which necessitates approaches beyond traditional computing paradigms to improve their operational efficiency. Leveraging the application-level insight of error resilience, we demonstrate how approximate computing (AxC) can significantly boost the efficiency of AI platforms and play a pivotal role in the broader adoption of AI-based applications and services. To this end, we present RaPiD, a multi-tera operations per second (TOPS) AI hardware accelerator core (fabricated at 14-nm technology) that we built from the ground-up using AxC techniques across the stack including algorithms, architecture, programmability, and hardware. We highlight the workload-guided systematic explorations of AxC techniques for AI, including custom number representations, quantization/pruning methodologies, mixed-precision architecture design, instruction sets, and compiler technologies with quality programmability, employed in the RaPiD accelerator.

The growing prevalence and computational demands of Artificial Intelligence (AI) workloads has led to widespread use of hardware accelerators in their execution. Scaling the performance of AI accelerators across generations is pivotal to their success in commercial deployments. The intrinsic error-resilient nature of AI workloads present a unique opportunity for performance/energy improvement through precision scaling. Motivated by the recent algorithmic advances in precision scaling for inference and training, we designed RaPiD1, a 4-core AI accelerator chip supporting a spectrum of precisions, namely, 16 and 8-bit floating-point and 4 and 2-bit fixed-point. The 36mm2 RaPiD chip fabricated in 7nm EUV technology delivers a peak 3.5 TFLOPS/W in HFP8 mode and 16.5 TOPS/W in INT4 mode at nominal voltage. Using a performance model calibrated to within 1% of the measurement results, we evaluated DNN inference using 4-bit fixed-point representation for a 4-core 1 RaPiD chip system and DNN training using 8-bit floating point representation for a 768 TFLOPs AI system comprising 4 32-core RaPiD chips. Our results show INT4 inference for batch size of 1 achieves 3 - 13.5 (average 7) TOPS/W and FP8 training for a mini-batch of 512 achieves a sustained 102 - 588 (average 203) TFLOPS across a wide range of applications.

Low-precision computation is the key enabling factor to achieve high compute densities (T0PS/W and T0PS/mm2) in AI hardware accelerators across cloud and edge platforms. However, robust deep learning (DL) model accuracy equivalent to high-precision computation must be maintained. Improvements in bandwidth, architecture, and power management are also required to harness the benefit of reduced precision by feeding and supporting more parallel engines to achieve high sustained utilization and optimize performance within a given product power envelope. In this work, we present a 4-core AI chip in 7nm EUV technology that exploits cutting-edge algorithmic advances for iso-accurate models in low-precision training and inference [1, 2] and aggressive circuit/architecture optimization to achieve leading-edge power-performance. The chip supports fp16 (DLFIoat16 [8]) and hybrid-fp8(hfp8) [1] formats for training and inference of DL models, as well as int4 and int2 formats for highly scaled inference.

A processor core is presented for AI training and inference products. Leading-edge compute efficiency is achieved for robust fp16 training via efficient heterogeneous 2-D systolic array-SIMD compute engines leveraging compact DLFloat16 FPUs. Architectural flexibility is maintained for very high compute utilization across neural network topologies. A modular dual-corelet architecture with a shared scratchpad and a software-controlled network/memory interface enables scalability to many-core SoCs and large-scale systems. The 14nm AI core achieves fp16 peak performance of 3.0 TFLOPS at 0.62V and 1.4 TFLOPS/W at 0.54V.

This letter presents a multi-TOPS AI accelerator core for deep learning training and inference. With a programmable architecture and custom ISA, this engine achieves >90% sustained utilization across the range of neural network topologies by employing a dataflow architecture to provide high throughput and an on-chip scratchpad hierarchy to meet the bandwidth demands of the compute units. A custom 16b floating point (fp16) representation with 1 sign bit, 6 exponent bits, and 9 mantissa bits has also been developed for high model accuracy in training and inference as well as 1b/2b (binary/ternary) integer for aggressive inference performance. At 1.5 GHz, the AI core prototype achieves 1.5 TFLOPS fp16, 12 TOPS ternary, or 24 TOPS binary peak performance in 14-nm CMOS.

The combination of growth in compute capabilities and availability of large datasets has led to a re-birth of deep learning. Deep Neural Networks (DNNs) have become state-of-the-art in a variety of machine learning tasks spanning domains across vision, speech, and machine translation. Deep Learning (DL) achieves high accuracy in these tasks at the expense of 100s of ExaOps of computation; posing significant challenges to efficient large-scale deployment in both resource-constrained environments and data centers. One of the key enablers to improve operational efficiency of DNNs is the observation that when extracting deep insight from vast quantities of structured and unstructured data the exactness imposed by traditional computing is not required. Relaxing the "exactness" constraint enables exploiting opportunities for approximate computing across all layers of the system stack. In this talk we present a multi-TOPS AI core [3] for acceleration of deep learning training and inference in systems from edge devices to data centers. We demonstrate that to derive high sustained utilization and energy efficiency from the AI core requires ground-up re-thinking to exploit approximate computing across the stack including algorithms, architecture, programmability, and hardware. Model accuracy is the fundamental measure of deep learning quality. The compute engine precision in our AI core is carefully calibrated to realize significant reduction in area and power while not compromising numerical accuracy. Our research at the DL algorithms/applications-level [2] shows that it is possible to carefully tune the precision of both weights and activations to as low as 2-bits for inference and was used to guide the choices of compute precision supported in the architecture and hardware for both training and inference. Similarly, distributed DL training's scalability is impacted by the communication overhead to exchange gradients and weights after each mini-batch. Our research on gradient compression [1] shows by selectively sending gradients larger than a threshold, and by further choosing the threshold based on the importance of the gradient we achieve achieve compression ratio of 40X for convolutional layers, and up to 200X for fully-connected layers of the network without losing model accuracy. These results guide the choice of interconnection network topology exploration for a system of accelerators built using the AI core. Overall, our work shows how the benefits from exploiting approximation using algorithm/application's robustness to tolerate reduced precision, and compressed data communication can be combined effectively with the architecture and hardware of the accelerator designed to support these reduced-precision computation and compressed data communication. Our results demonstate improved end-to-end efficiency of the DL accelerator across different metrics such as high sustained TOPs, high TOPs/watt and TOPs/mm2 catering to different operating environments for both training and inference.

A multi-TOPS AI core is presented for acceleration of deep learning training and inference in systems from edge devices to data centers. With a programmable architecture and custom ISA, this engine achieves >90% sustained utilization across the range of neural network topologies by employing a dataflow architecture and an on-chip scratchpad hierarchy. Compute precision is optimized at 16b floating point (fp 16) for high model accuracy in training and inference as well as 1b/2b (bi-nary/ternary) integer for aggressive inference performance. At 1.5 GHz, the AI core prototype achieves 1.5 TFLOPS fp 16, 12 TOPS ternary, or 24 TOPS binary peak performance in 14nm CMOS.

3-D integration using through-silicon-vias (TSVs) is emerging as one of the key technology options for continued miniaturization. However, because of increased device and current density, the reliability of the 3-D power grid and its integrity must be studied and analyzed. Due to the geometry of TSVs and connections to the global power grid, significant current crowding can occur. Current densities at these connections can be much higher than the expected average values, so extra care is required for accurate analysis. In prior work, TSVs are modeled as single resistors along with power grid wire segments. Such models do not capture detailed current density distribution and may miss hotspots associated with current crowding. This paper studies current crowding and its impact on 3-D power grid integrity. First, we explore the current density distribution within a TSV and its connections to the global chip power grid. Second, we implement simple TSV models to obtain current density distributions within a TSV and its local environment. These models are checked for accuracy by comparing with models simulated using finite element modeling methods. Finally, the simple TSV models are integrated with the global power grid for detailed chip-scale power analysis.

3D chip stacking technology has the potential to enable increased system performance through integration of heterogeneous system components, such as accelerators and high density memory, as well as through increased area for tightly integrated processor components in multi-core systems. This paper describes the design, measurements and a thermal modeling methodology used to achieve accurate 3D thermal model-to-hardware correlation for two and three layer 3D high-power “logic” stacks.

This paper presents a design and modeling methodology of vertical interconnects for three-dimensional integration (3DI) applications. Compact semi-analytical wideband circuit level models have been developed based on explicit expressions. The pronounced frequency dependent silicon substrate induced dispersion and loss effects are considered, as well as skin and proximity effects. The models have been verified against numerical computations (full wave HFSS and quasi-static Q3D solvers). A dedicated test site has been designed for broadband characterization (from 1 MHz up to 110 GHz) of TSVs within a dense farm.

The IBM z14 design was built with the 14-nm high-performance silicon-on-insulator (SOI) technology of GLOBALFOUNDRIES. This was the first technology node after IBM transitioned from its integrated fabrication facility to operating in a fabless environment, driving significant changes to design processes and methodology. In addition to this partnership, the 14-nm technology introduced significant changes relative to previous technology nodes, including the introduction of fin-shaped field-effect transistors, the use of double patterning for the lowest back-end-of-line layers, and the introduction of middle-of-line layers to exploit contact layers for local interconnects. This combination of technical and business challenges required numerous large-scale innovations for our design, design team, and design methodologies. In this paper, we provide a survey of these innovations, including the fin-based standard cell image, deeply scaled SOI self-heating/electromigration verification, routing strategies to handle double-patterning with interlayer via awareness, fill automation to enable simultaneous design of multiple layers of hierarchy, and high-performance array design with the voltage and noise limitations of the 14-nm technology node.

Due to the large geometry of through-silicon-vias (TSVs) and their connections to the power grid, significant current crowding can occur in 3D ICs. Prior works model TSVs and power wire segments as single resistors, which cannot capture the detailed current distribution and may miss trouble spots associated with current crowding. This paper studies DC current crowding and its impact on 3D power integrity. First, we explore the current density distribution within a TSV and its power wire connections. Second, we build and validate effective TSV models for current density distributions. Finally, these models are integrated with global power wires for detailed chip-scale power grid analysis.

It is important to understand how to deliver power into 3D heterogeneous systems, which require different power supplies for different components (e.g., digital, analog, mixed-signal, MEMS parts). This paper reports on a study for a simplified case, where the two power supplies with different voltage levels are uniformly distributed through a TSV-based 3D system. In addition to intrinsic voltage losses along the interconnections, we investigated for the first time the power noises due to electrical couplings between the multiple power supplies. We evaluated the through-silicon-via (TSV)-based 3D power delivery networks by combining the electromagnetic (EM) and SPICE simulations. With this hybrid approach, we first partitioned the 3D system into a number of elements, extracted the RLGC parasitics for each decomposed physical element, and imported them into an assembled system-level equivalent circuit to characterize the 3D power distribution networks.

While three-dimensional (3D) technology has several advantages for power delivery, an integrated chip-level, interposer-level, and package-level power distribution network in through-silicon-via (TSV)-based 3D system has to be modeled and evaluated. This paper reports on modeling of power delivery into 3D chip stacks on a silicon interposer/packaging substrate using a novel hybrid approach, i.e., combining the electromagnetic (EM) and analytic simulations. We intentionally partition the real stack-up structure of a 3D power network into separate components, i.e., package vias and traces, C-4 solders, interposer TSVs and planar wires, μ-C4 solders, chip TSVs, and on-chip power grids with node capacitors, decoupling capacitors and active current loads. All the passive RLGCs for each component are extracted using an EM simulation tool at a given working frequency point. We then assemble all the components back into a corresponding equivalent circuit model with those EM extracted RLGC values, thus to analyze the supply voltage (V dd )variation over time for 3D systems in a manner of maximum accuracy and efficiency.

Decoupling capacitors are essential to reduce high transient current noise and to provide a low impedance power delivery path. 3D technology has several advantages for power delivery, and this work investigates the impacts of decoupling capacitors on through-silicon-via (TSV)-based 3D power networks using a novel hybrid modeling approach, i.e., combining electromagnetic (EM) and SPICE simulations. We first partitioned a 3D system into a number of components, extracted the RLGC parasitics for each decomposed physical element, and imported them into an assembled system-level equivalent circuit. Through comparing and analyzing the effectiveness of several decoupling strategies, design tradeoffs are made in selecting the proper values and placement of decoupling capacitors to ensure optimal power distribution solution in 3D architectures.

3D integration (3DI) holds promise for improved performance of integrated systems by increasing interconnect bandwidth [1]. A processor stacked with cache memory is one potential application of 3DI [2]. This work describes the design and operation of a prototype of a 3D system, constructed by stacking a memory layer, built with eDRAM [3] and logic blocks from the IBM Power7™ processor L3 cache, and a “processor proxy” layer in 45nm CMOS technology [4] enhanced to include through-silicon vias (TSVs) [5]. Unlike the previously reported 3D eDRAM [6], the 3D stack described here is constructed using 50μm pitch μC4's joining the front side of one thick chip to TSV connections on the back side of a thinned chip. TSVs are formed of Cu-filled vias that are ∼20μm in diameter and

For an epitaxial c -axis oriented slightly oxygen deficient YBaCuO film, showing a positive Seebeck coefficient around 100 K, the Nernst effect has been measured in the mixed state for magnetic fields up to 9 T. The transport entropy calculated in the usual way is about an order of magnitude smaller than for a similar but fully oxygenated film with a negative Seebeck coefficient near 100 K. This can be explained by the enhanced decoupling of the CuO 2 planes in the slightly oxygen deficient material, resulting in a strong departure of the vortices (“pancake vortices”) from the standard Abrikosov type.

The authors have fabricated and characterized interdigitated silicon metal-semiconductor-metal photodetectors (MSM-PDs) using a self-aligned PtSi Schottky barrier metallurgy and SiO/sub 2/ passivation in a VLSI-compatible process. These MSM-PDs are optimal for high speed applications at wavelengths >

The fourth-generation POWER processor chip contains 170M transistors and includes 2 microprocessor cores, shared L2, directory for an off-chip L3, and all logic needed to interconnect multiple chips to form an SMP. It is implemented in a 0.18 /spl mu/m SOI technology, with 7 layers of Cu interconnect, and functions in systems at 1.1 GHz, and dissipates 115 W at 1.5 V.

We have made the first direct measurements of the critical current density of individual grain boundaries and their adjoining grains in ${\mathrm{Y}}_{1}$${\mathrm{Ba}}_{2}$${\mathrm{Cu}}_{3}$${\mathrm{O}}_{7\mathrm{\ensuremath{-}}\mathrm{\ensuremath{\delta}}}$ superconducting films. The critical current of the boundary was always found to be less than either of the two grains. The magnetic field dependence of the critical current of the grain boundaries indicates that they are comprised of regions of weak and strong Josephson coupling. A single grain boundary was also used to build and operate a simple dc SQUID. Current-voltage curves of a grain-boundary junction n the finite voltage range show subgap structure.

This paper reports on progress made at IBM in the fabrication of HiTc dc SQUIDs (Superconducting QUantum Interference Devices) 1,2 . We have fabricated SQUIDs from polycrystalline YBa 2 Cu 3 O y samples on MgO substrates 3 and from epitaxial films on SrTiO 3 (100) substrates 4 . The films were patterned using either ion-implanting to selectively convert the superconductor into an insulator without actually removing any material 1,5 , laser ablation to pattern a loop into the film by removing material 6 , or a conventional ion milling process to selectively remove material using a photoresist mask. In the case of the polycrystalline films, the naturally occurring grain boundaries provide the weak link or Josephson junction behavior in a stripe that is much larger in size than the superconducting coherence length. We have measured the noise amplitude of one such SQUID made on MgO to be about1 × 10 -5 Φ O /Hz 1/2 at 1000 Hz and 40 K. Loops formed from true epitaxial films containing many twin boudaries will not operate as dc SQUIDs unless the films are damaged across each link. A SQUID formed in this way operated in liquid nitrogen and had an estimated white noise performance of1 × 10 -3 Φ 0 /Hz 1/2 at 74 K.

Measurements of electrical resistance of thin films of the high‐temperature superconductor Y1Ba2Cu3Ox have been carried out between 100 and 800 °C, and in the range between roughly 0.1 and 1 atm of ambient oxygen. Results are reported for both quick and slow heating, and also for sudden changes in oxygen partial pressure. At constant temperature we observe that the response to the oxygen partial pressure is both quick and reversible. We show evidence that quick heating is preferable, and we also observe features in the resistance similar to those reported in bulk samples and interpreted as an order–disorder transition of oxygen in the one‐dimensional Cu‐O chains.

It is explicitly shown that the spatial distribution of the energy gap in a superconducting tunnel junction can be determined quantitatively. This is verified experimentally using semiconductor-insulator-semiconductor tunnel junctions where gap inhomogeneity is de- liberately induced with patterned laser illumination.

Micron‐wide lines of high Tc Y‐Ba‐Cu‐O have been successfully patterned by ablating the films with a pulsed excimer laser. The high Tc films are mounted onto a computer‐controlled stepping stage and irradiated with a demagnified image of a variable‐size rectangular aperture. This technique has been used for fabricating features ranging from several centimeters in length to submicron in width without any degradation in Tc. For example, a superconducting microstructure of Y‐Ba‐Cu‐O, nominally 1 μm wide and 2.5 μm long, with a Tc (R=0) of 88 K and a Jc of 5×104 A/cm2 at 4.2 K was fabricated.

High Tc Y‐Ba‐Cu‐O films have been prepared by dc magnetron sputtering of metal alloy targets. To circumvent the negative ion effect, two alloy targets, YCu and BaCu, are sputtered in an argon atmosphere with an oxygen spray near the substrate. Films deposited on sapphire with onsets at 92 K and a 6° transition width (10–90%) have been achieved using this technique. These films have been successfully patterned with the technique of laser ablation.

The effect of uniaxial strain e on superconducting transition temperature Tc of thin YBa2Cu3O7 film is presented. The value of TC increases linearly with compressional strain with a slope of -275K/e. Under tensile strain, Tc initially decreases to a value ∼ 0.3K below the zero-strain value and then remains constant with additional strain. In compression, the logarithmic strain derivative of Tc, (1/Tc)(dTc/de), is -3 per unit strain which is slightly less than 4–6 per unit strain observed in conventional superconductors.

Picosecond electrical pulses with well resolved frequency components into the terahertz region are ideally suited for the characterization of VLSI electronics, both for the devices themselves and for the interconnect wiring. In this paper, we describe the technology we employ to photoconductively generate and measure these ultrafast electrical signals./1/ A particular emphasis was given to the compatibility with existing VLSI device technology. Although other measurement techniques /2,3/ have shown very good sensitivity and time resolution, the photoconductive sampling technique is particularly attractive for silicon device characterization with a comparable sensitivity and time resolution /4/ and a better amplitude calibration and reproducibility as the sampling geometry is defined by the lithography itself. Impedance matching is facilitated while keeping identical geometries between the device site and the generating and sampling sites. We also present results on the optoelectronic characterization of a 1μm gate GaAs MESFET. We will not address in this paper the problem of circuit testing which has some stringent additional requirements, in particular that of submicron spatial resolution and of wide field addressing capability. These imperatives may ultimately limit the use of purely optical techniques for the functional testing of high speed circuits at their internal nodes.

Laser ablation using focussed excimner light has been found to be an effective technique for the direct, maskless patterning of High Tc superconductor films. The apparatus used in the work consists of an adjustable aperture which is illuminated by a 193 nm excimer laser beam. The aperture is imaged by a microscope onto the substrate into an area as large as 200 × 200 to an area as small as 2 × 2 square microns. The superconductor films are placed on a 0.5 micron resolution, computer controlled x-y stage. Laser fluences up to 5 J/cm2 and repetition rates of 10 Hz were found to be sufficient to ablate (pattern) the High Tc films with no apparent effect on their superconducting properties as evidenced by the films′ resistance as a function of temperature (R vs T) before and after patterning.The technique and apparatus are flexible enough to allow patterning across entire substrates and/or the definition of isolated features as small as one micron. To date, the technique has been successfully used to pattern a wide variety of films (based on material and thickness), to fabricate transmission lines several millimeters long, to pattern superconducting quantum interference devices (SQUIDs) which operate at liquid nitrogen temperatures, and to study the behavior of individual grain boundaries within films.

Thin film superconductors of the YBaCuO family have been fabricated using layer structures of Cu/BaO/Y2O3 deposited by electron-beam evaporation. Using layer thicknesses off the stoichiometric YBa2Cu3O7, we have formed superconducting films with zero resistance above 77°K using MgO and SrTiO3 substrates. The films were found to be composed of both the superconducting YBa2Cu3O7 phase, and the non-superconducting green Y2BaCuO7 phase. Dependence of transition temperatures on the annealing temperatures and the substrates has also been studied.

The superconducting properties of high-transition-temperature (high-Tc) copper-oxide superconductors depend sensitively on their microstructure. For example, the critical current density Jc of epitaxial YBa2Cu3O7 films can be several orders of magnitude higher than that of fine-grained polycrystalline films and ceramics1,2. Local variations in the size and distribution of features such as grains, grain boundaries, anisotropy and composition play an important role in determining the superconducting properties, and also profoundly affect the normal-state properties. A technique capable of probing superconducting properties with high spatial resolution—smaller than the grain size—is thus important not only for basic understanding, but also for device and microelectronics applications. Here we combine the techniques of micro-patterning3,4 and low-temperature scanning electron microscopy (LTSEM) 5,6 to obtain the first spatially resolved observation of local values of the critical current Ic in high-Tc polycrystalline films. Both the range of values of Ic and the spatial distribution of the local Ic can be determined with a resolution approaching 1 (µm. This new technique can be extended easily to the spatially resolved investigation of Tc distribution, flux pinning and flux flow6.