Search

The IBM z14 is the latest update in the storied history of IBM mainframes. Reliability, availability, security, and scalability are the foundation of the IBM mainframe line. System reliability and availability targets are in excess of 10 years, requiring rigorous chip characterization processes. In this paper, we discuss some of the many processes used to ensure that lifetime. An additional part of this reliability is power management (PM). The 5.2-GHz high-power design of the central processor chip requires advanced on-die PM capabilities to adapt to power intensive instruction streams. We also discuss a number of improvements to the critical path monitoring design used to manage power supply voltage droops, reducing response time and the impact on system performance. Finally, we compare a set of simulations and hardware results to validate our power fluctuation models.

The IBM z15 processor power management enhances several on-chip power management techniques over z14 processor with a specific focus on reducing response time for voltage droop management. The IBM z15 processor puts a specific emphasis on proactive voltage droop management strategy to reduce conservative static guard band that is added to the supply voltage in order to protect against worst-case voltage droops. The z15 processor relies on selected events from the earlier stages of a deep pipeline processor as indicators to predict sharp changes in the power consumption over a short period of time. The early information of the selected events allows to throttle the execution flow through the processor pipeline and prevents the sharp power change before it takes place and thus reduces the voltage droop. In z15, as one of the proactive schemes, we combine both the digital power-proxies, which are direct indicators of the processor activity and the Critical Path Monitors (CPMs) to give an earlier and proactive indication of voltage droop events. This proactive indication provides enough time for the throttle actuation circuits to prevent the voltage droop. CPMs act as real-time timing margin indicators, and power-proxies act to serve as the activity monitors.

Enterprise server processor designs, which operate at extreme high frequencies and power envelopes, depend critically on power supply noise mitigation techniques. With supply voltage scaling, very high current draws, and broad usage of clock gating, advanced solutions are needed for next-generation products to minimize droop mitigation response time, which can be defined as the latency from when a dangerous droop begins until a countermeasure is effective.

Successful power supply noise mitigation requires a system-level approach that includes design and modeling of the mitigation circuits with the power delivery network (PDN) on the chip, the chip module, the backplane, and the voltage regulator module (VRM). Traditionally, periodic square-wave activity patterns with all cores in sync, which yield low-frequency (LF) or mid-frequency (MF) impedance peaks associated with the backplane and chip/module, respectively, are considered to give rise to the worst case power supply noise. However, voltage droops that are both deeper and faster at a single victim core are created when cores change activity in more complicated patterns, termed as perfect storms in this work. These patterns excite high-frequency (HF) modes that are not stimulated when all cores switch simultaneously, and require an accurate model of the packaged chip, including effective core-to-core inductances due to currents traveling between cores through low-resistance module planes.

This work describes the circuit and physical design implementation of the processor chip (CP), level-4 cache chip (SC), and the multi-chip module at the heart of the EC12 system. The chips were implemented in IBM's high-performance 32nm high-k/metal-gate SOI technology. The CP chip contains 6 super-scalar, out-of-order processor cores, running at 5.5 GHz, while the SC chip contains 192 MB of eDRAM cache. Six CP chips and two SC chips are mounted on a high-performance glass-ceramic substrate, which provides high-bandwidth, low-latency interconnections. Various aspects of the design are explored in detail, with most of the focus on the CP chip, including the circuit design implementation, clocking, thermal modeling, reliability, frequency tuning, and comparison to the previous design in 45nm technology.

This paper describes the circuit and physical design features of the z196 processor chip, implemented in a 45 nm SOI technology. The chip contains 4 super-scalar, out-of-order processor cores, running at 5.2 GHz, on a die with an area of 512 mm2 containing an estimated 1.4 billion transistors. The core and chip design methodology and specific design features are presented, focusing on techniques used to enable high-frequency operation. In addition, chip power, IR drop, and supply noise are discussed, being key design focus areas. The chip's ground-breaking RAS features are also described, engineered for maximum reliability and system stability.

Voltage noise characterization is an essential aspect of optimizing the shipped voltage of high-end processor based systems. Voltage noise, i.e. Variations in the supply voltage due to transient fluctuations on current, can negatively affect the robustness of the design if it is not properly characterized. Modeling and estimation of voltage noise in a pre-silicon setting is typically inadequate because it is difficult to model the chip/system packaging and power distribution network (PDN) parameters very precisely. Therefore, a systematic, direct measurement-based characterization of voltage noise in a post-silicon setting is mandatory in validating the robustness of the design. In this paper, we present a direct measurement-based voltage noise characterization of a state-of-the-art mainframe class multicoreprocessor. We develop a systematic methodology to generate noise stress marks. We study the sensitivity of noise in relation to the different parameters involved in noise generation: (a) stimulus sequence frequency, (b) supply current delta, (c) number of noise events and, (d) degree of alignment or synchronization of events in a multi-core context. By sensing per-core noise in amulti-core chip, we characterize the noise propagation across the cores. This insight opens up new opportunities for noise mitigation via workload mappings and dynamic voltage guard banding.

The new System z microprocessor chip (“CP chip”) features a high-frequency processor core running at 5.5GHz in a 32nm high-κ CMOS technology [1], using 15 levels of metal. This chip is a successor to the 45nm product [2], with significant improvements made to the core and nest (i.e. the logic external to the cores) in order to increase the performance and throughput of the design. Also, special considerations were necessary to ensure robust circuit operation in the high-κ technology used for implementation. As seen in the die photo, the chip contains 6 processor cores (compared to 4 cores in the 45nm version), and a large shared 48MB DRAM L3 cache. Each core includes a pair of data and instruction L2 SRAM caches of 1MB each. In addition, the chip contains a memory control unit (MCU), an I/O bus controller (GX), and two sets of interfaces to the L4 cache chips (also in 32nm technology). The CP chip occupies 598 mm2, contains about 2.75B transistors, and has 1071 signal IOs.

Considerations for comprehensive simulation and analysis of high-speed links in complex server systems are discussed in this work. A framework that considers the interactions between multiple signals and the power distribution network is described. In the last section, the nature of the interactions between power and signal domains is analyzed by means of a simplified scenario at board level.

The IBM zEnterprise z196 processor chip is an energy efficient high-frequency, high-performance design that implements 4 processor cores optimized for maximum single-thread performance. Chip energy efficiency is improved by 25% compared to the previous 65nm design, which enables the processor chip to run at product frequency of 5.2 GHz, providing a significant performance boost for the z196 system. This paper discusses the enablement of a high frequency and high performance design with a focus on energy consumption challenges and solutions. Design tradeoffs for processor speed, performance and energy consumption were optimized during project concept phase and practiced through detailed implementation stages on all aspects of the processor design. Various high speed circuit techniques were deployed to achieve the high frequency goal and improve overall energy efficiency. A comprehensive power methodology was developed to calculate leakage and dynamic power dissipation at various workloads. Sustained thermal power as well as instantaneous peak power was analyzed and worked on throughout the entire design process. The final design stays within the system power constraints and achieves a 5.2Ghz product frequency.

The microprocessor chip for the IBM zEnterprise 196 (z 196) system is a high-frequency, high-performance design that adds support for out-of-order instruction execution and increases operating frequency by almost 20% compared to the previous 65nm design, while still fitting within the same power envelope. Despite the many difficult engineering hurdles to be overcome, the design team was able to achieve a product frequency of 5.2GHz, providing a significant performance boost for the new system.

The two chips at the heart of the IBM z13™ system include a processor chip (referred to as the CP or Central Processor chip) and an L4 (Level 4) cache chip (referred to as the SC or System Controller chip), each 678 mm2 in area. The CP and SC chips were implemented with approximately 4 billion (4 × 109) and 7.1 billion transistors, respectively, in IBM's 22-nm SOI (silicon-on-insulator) technology, supporting eDRAM (embedded dynamic random access memory), and with up to 17 levels of metal available. In this paper, we discuss aspects of the circuit and physical design of these chips, including both digital logic and custom array implementation. In addition, we describe the design analysis methodology, along with some of the checks needed to ensure a robust, reliable, and high-frequency product.

The power management strategy adopted for the IBM z13™ processor chip (referred to as the CP or Central Processor chip) is guided by three basic principles: (a) controlling the peak power consumpti...

An on chip circuit model for accurate mid-frequency simultaneous switching noise (SSN) prediction has been studied in this paper. Voltage controlled switching resistors, voltage controlled leakage resistors have been proposed to represent the switching activities and leakage of a chip. Empirical formulas are presented for the calculations of resistors. The on chip power grid impacts on mid-frequency SSN as well as the clock frequency noise have been analyzed. For mid-frequency SSN prediction, we don't have to include on chip power grid in the model except an accurate intrinsic capacitance extraction. However, for clock-frequency noise, we need the on chip power grid characteristics.

In this paper, we describe the first- and second-level system packaging structure of the IBM zEnterprise® 196 (z196) enterprise-class server. The design point required a more than 50% overall increase in system performance (in millions of instructions per second) in comparison to its predecessor. This resulted in a new system design that includes, among other things, increased input/output bandwidth, more processors with higher frequencies, and increased current demand of more than 2,000 A for the six processor chips and two cache chips per multichip module. To achieve these targets, we implemented several new packaging technologies. The z196 enterprise-class server uses a new differential memory interface between the processor chips and custom-designed server memory modules. The electrical power delivery system design follows a substantially new approach using Vicor Factor Power® blocks, which results in higher packaging integration density and minimized package electrical losses. The power noise decoupling strategy was changed because of the availability of deep-trench technology on the new processor chip generation.

This paper describes the system packaging and technologies of the IBM System z10™ high-end Enterprise Class server. This machine exceeds the multiprocessor performance of the previous system by 50%. A new generation of the IBM Elastic Interface was developed in order to maintain the increased interconnect signal speed of up to 2.93 Gb/s. Power control and power delivery to the multicore processors were a special challenge for the server packaging because of the high currents and the high number of voltage domains.