Search

FPGAs are rarely mentioned when discussing the implementation of large machine learning applications, such as Large Language Models (LLMs), in the data center. There has been much evidence showing that single FPGAs can be competitive with GPUs in performance for some computations, especially for low latency, and often much more efficient when power is considered. This suggests that there is merit to exploring the use of multiple FPGAs for large machine learning applications. The challenge with using multiple FPGAs is that there is no commonly-accepted flow for developing and deploying multi-FPGA applications, i.e., there are no tools to describe a large application, map it to multiple FPGAs and then deploy the application on a multi-FPGA platform. In this paper, we explore the feasibility of implementing large transformers using multiple FPGAs by developing a scalable multi-FPGA platform and some tools to map large applications to the platform. We validate our approach by designing an efficient multi-FPGA version of the I-BERT transformer and implement one encoder using six FPGAs as a working proof-of-concept to show that our platform and tools work. Based on our proof-of-concept prototype and the estimations of performance using the latest FPGAs compared to GPUs, we conclude that there can be a place for FPGAs in the world of large machine learning applications. We demonstrate a promising first step that shows that with the right infrastructure and tools it is reasonable to continue to explore the possible benefits of using FPGAs for applications such as LLMs., Comment: 33 pages, 24 figures

Datacenters of today have maintained the same architecture for decades using the server as the primary building block. However, this traditional approach suffers from under-utilization of its resources, often caused by over-allocating these resources when deploying applications to accommodate worst-case scenarios. Specifically, servers can quickly drain their over-allocated memory resources while their CPUs are not fully utilized. This problem gives rise to a different school of thought, where resources are disaggregated instead of tightly bound to servers. This can address the utilization problem by allowing each type of resource to be allocated, utilized and freed separately as required. New high performance communication protocols, like CXL, could pave the way for practical implementations of resource disaggregation. In this article, we argue it is time to reconsider the datacenter architecture as a whole. We present our vision for a disaggregated datacenter aided by well-established computer architecture design methodologies.

High pressure x-ray diffraction up to 30 GPa and resonant emission x-ray spectroscopy and partial fluorescence yield x-ray absorption spectroscopy up to 52 GPa were used to study how the structural and electronic properties of UTe$_2$ evolve with pressure at room temperature. An orthorhombic to tetragonal phase transition was observed to occur between 5 and 7 GPa, with a large volume collapse of nearly 11% and a nearest U-U distance increase by about 4%. This lower to higher symmetry transition suggests less 5f electron participation in bonding when the weakly correlated superconducting phase in the tetragonal structure of UTe$_2$ appears. Beyond 7 GPa, no new structural transitions were found up to 30 GPa. The resonant x-ray emission spectra clearly demonstrate an intermediate valence of U, nearly +3.74 at 1.8 GPa and room temperature, and reveal that the U valence shifts towards 4+, passes through a peak at 2.8 GPa, and then decreases towards 3+ and settles down to a nearly constant value above 15 GPa. These experiments reveal that some fundamental structural and valence changes occur in UTe2 at relatively low pressures, which could be responsible for the interplay between unconventional superconductivity, magnetic ordering, and weakly correlated superconductivity that is manifested in the temperature-pressure phase diagram of UTe2., Comment: 20 pages, 8 figures, 1 table

In today's data centers, the performance of interconnects plays a pivotal role. However, many of the underlying technologies for these interconnects have a history of several decades and existed long before data centers came into being.To better cater to the requirements of data center networks, particularly in the context of intra-rack communication, we have developed a new interconnect. This interconnect is based on a lossless link layer protocol, named RIFL. In this work, we designed and implemented RIFL Layer 2, a scalable network that supports up to multi-hundred Gbps communication. RIFL Layer 2 includes the RIFL switch and RIFL NIC. By utilizing a simple Batcher Banyan and iSLIP RIFL switch, we effectively keep the typical intra-rack latency under 400 nanoseconds. Moreover, for a 32-port 100Gbps network, under both Bernoulli arrival and bursty arrival traffic patterns, we ensure that the 99\% tail latency does not exceed 12microseconds.

More and more latency-sensitive services and applications are being deployed into the data center. Performance can be limited by the high latency of the network interconnect. Because the conventional network stack is designed not only for LAN, but also for WAN, it carries a great amount of redundancy that is not required in a data center network. This paper introduces the concept of a three-layer protocol stack that can fulfill the exact demands of data center network communications. The detailed design and implementation of the first layer of the stack, which we call RIFL, is presented. A novel low latency in-band hop-by-hop re-transmission protocol is proposed and adopted in RIFL, which guarantees lossless transmission in a data center environment. Experimental results show that RIFL achieves 110 nanoseconds point-to-point latency on 10-meter Active Optical Cables, at a line rate of 112 Gbps. RIFL is a multi-lane protocol with scalable throughput up to multi-hundred gigabits per second. It can be the enabler of low latency, high throughput, flexible, scalable, and lossless data center networks., Comment: 15 pages, 9 figures, journal

The strongly correlated material FeSi exhibits several unusual thermal, magnetic, and structural properties under varying pressure-temperature (P-T) conditions. It is a potential thermoelectric alloy and a materials of several geological implications as a possible constituent at the Earth's core mantle boundary (CMB). The phase transition behavior and lattice dynamics of FeSi under different P-T conditions remain elusive. A previous theoretical work predicted a pressure induced B20-B2 transition at ambient temperature, yet the transition is only observed at high P-T conditions in the experiments. Furthermore, the closing of the electronic gap due to a dramatic renormalization of the electronic structure and phonon anomalies has been reported based on density function calculations. In this study we have performed high pressure powder diffraction and Nuclear Resonant Inelastic X-ray Scattering (NRIXS) measurements up to 120 GPa to understand the phase stability and the lattice dynamics. Our study shows evidence for a nonhydrostatic stress induced B20-B2 transition in FeSi around 36 GPa for the first time. The Fe partial phonon density of states (PDOS) and thermal parameters were derived from NRIXS up to 120 GPa with the density function theoretical (DFT) calculations. These calculations further predict and are consistent with pressure-induced metallization and band gap closing around 12 GPa., Comment: 8 pages, 6 figures and supplementary information

Valence transition could induce structural, insulator-metal, nonmagnetic-magnetic and superconducting transitions in rare-earth metals and compounds, while the underlying physics remains unclear due to the complex interaction of localized 4f electrons as well as their coupling with itinerant electrons. The valence transition in the elemental metal europium (Eu) still has remained as a matter of debate. Using resonant x-ray emission scattering and x-ray diffraction, we pressurize the states of 4f electrons in Eu and study its valence and structure transitions up to 160 GPa. We provide compelling evidence for a valence transition around 80 GPa, which coincides with a structural transition from a monoclinic (C2/c) to an orthorhombic phase (Pnma). We show that the valence transition occurs when the pressure-dependent energy gap between 4f and 5d electrons approaches the Coulomb interaction. Our discovery is critical for understanding the electrodynamics of Eu, including magnetism and high-pressure superconductivity.

This work presents a heterogeneous communication library for clusters of processors and FPGAs. This library, Shoal, supports the Partitioned Global Address Space (PGAS) memory model for applications. PGAS is a shared memory model for clusters that creates a distinction between local and remote memory access. Through Shoal and its common application programming interface for hardware and software, applications can be more freely migrated to the optimal platform and deployed onto dynamic cluster topologies. The library is tested using a thorough suite of microbenchmarks to establish latency and throughput performance. We also show an implementation of the Jacobi iterative method that demonstrates the ease with which applications can be moved between platforms to yield faster run times. Through this work, we have demonstrated the feasibility of using a PGAS programming model for multi-node heterogeneous platforms.

We report a pressure-induced phase transition in the frustrated kagom\'e material jarosite at ~45 GPa, which leads to the disappearance of magnetic order. Using a suite of experimental techniques, we characterize the structural, electronic, and magnetic changes in jarosite through this phase transition. Synchrotron powder X-ray diffraction and Fourier transform infrared spectroscopy experiments, analyzed in aggregate with the results from density functional theory calculations, indicate that the material changes from a R-3m structure to a structure with a R-3c space group. The resulting phase features a rare twisted kagom\'e lattice in which the integrity of the equilateral Fe3+ triangles persists. Based on symmetry arguments we hypothesize that the resulting structural changes alter the magnetic interactions to favor a possible quantum paramagnetic phase at high pressure., Comment: Manuscript and Supplement included

We report the results of high pressure x-ray diffraction, x-ray absorption, and electrical transport measurements of Kondo insulator Ce$_3$Bi$_4$Pt$_3$ up to 42 GPa, the highest pressure reached in the study of any Ce-based KI. We observe a smooth decrease in volume and movement toward intermediate Ce valence with pressure, both of which point to increased electron correlations. Despite this, temperature-dependent resistance data show the suppression of the interaction-driven ambient pressure insulating ground state. We also discuss potential ramifications of these results for the predicted topological KI state., Comment: 8 pages, 5 figures

Spin crossover plays a central role in the structural instability, net magnetic moment modification, metallization, and even in superconductivity in corresponding materials. Most reports on the pressure-induced spin crossover with a large volume collapse so far focused on compounds with single transition metal. Here we report a comprehensive high-pressure investigation of a mixed Fe-Mn perovskite La2FeMnO6. Under pressure, the strong coupling between Fe and Mn leads to a combined valence/spin transition: Fe3+(S = 5/2) to Fe2+(S = 0) and Mn3+(S = 2) to Mn4+(S = 3/2), with an isostructural phase transition. The spin transitions of both Fe and Mn are offset by ~ 20 GPa of the onset pressure, and the lattice collapse occurs in between. Interestingly, Fe3+ ion shows an abnormal behavior when it reaches a lower valence state (Fe2+) accompanied by a + 0.5 eV energy shift in Fe K-absorption edge at 15 GPa. This process is associated with the charge-spin-orbital state transition from high spin Fe3+ to low spin Fe2+, caused by the significantly enhanced t2g-eg crystal field splitting in the compressed lattice under high pressure. Density Functional Theory calculations confirm the energy preference of the high-pressure state with charge redistribution accompanied by spin state transition of Fe ions. Moreover, La2FeMnO6 maintains semiconductor behaviors even when the pressure reached 144.5 GPa as evidenced by the electrical transport measurements, despite the huge resistivity decreasing 7 orders of magnitude compared with that at ambient pressure. The investigation carried out here demonstrates high flexibility of double perovskites and their good potentials for optimizing the functionality of these materials., Comment: Physical Review B in press

We studied the crystal structure and spin state of MnP under high pressure with synchrotron X-ray diffraction and X-ray emission spectroscopy. MnP has an exceedingly strong anisotropy in compressibility, with the primary compressible direction along the b axis of the Pnma structure. X-ray emission spectroscopy reveals a pressure-driven quenching of the spin state in MnP. Firstprinciples calculations suggest that the strongly anisotropic compression behavior significantly enhances the dispersion of the Mn d-orbitals and the splitting of the d orbital levels compared to the hypothetical isotropic compression behavior. Thus, we propose spin quenching results mainly from the significant enhancement of the itinerancy of d electrons and partly from spin rearrangement occurring in the split d-orbital levels near the Fermi level. This explains the fast suppression of magnetic ordering in MnP under high pressure. The spin quenching lags behind the occurrence of superconductivity at ~8 GPa implying that spin fluctuations govern the electron pairing for superconductivity., Comment: 17 pages, 6 figures, accepted by New Journal of Physics

Convolutional Neural Networks (CNNs) have gained significant traction in the field of machine learning, particularly due to their high accuracy in visual recognition. Recent works have pushed the performance of GPU implementations of CNNs to significantly improve their classification and training times. With these improvements, many frameworks have become available for implementing CNNs on both CPUs and GPUs, with no support for FPGA implementations. In this work we present a modified version of the popular CNN framework Caffe, with FPGA support. This allows for classification using CNN models and specialized FPGA implementations with the flexibility of reprogramming the device when necessary, seamless memory transactions between host and device, simple-to-use test benches, and the ability to create pipelined layer implementations. To validate the framework, we use the Xilinx SDAccel environment to implement an FPGA-based Winograd convolution engine and show that the FPGA layer can be used alongside other layers running on a host processor to run several popular CNNs (AlexNet, GoogleNet, VGG A, Overfeat). The results show that our framework achieves 50 GFLOPS across 3x3 convolutions in the benchmarks. This is achieved within a practical framework, which will aid in future development of FPGA-based CNNs.

Resonant x-ray emission spectroscopy (RXES) was used to determine the pressure dependence of the f-electron occupancy in the Kondo insulator SmB6. Applied pressure reduces the f-occupancy, but surprisingly, the material maintains a significant divalent character up to a pressure of at least 35 GPa. Thus, the closure of the resistive activation energy gap and onset of magnetic order are not driven by stabilization of an integer valent state. Over the entire pressure range, the material maintains a remarkably stable intermediate valence that can in principle support a nontrivial band structure.

This paper introduces an effort to incorporate reconfigurable logic (FPGA) components into a software programming model. For this purpose, we have implemented a hardware engine for remote memory communication between hardware computation nodes and CPUs. The hardware engine is compatible with the API of GASNet, a popular communication library used for parallel computing applications. We have further implemented our own x86 and ARMv7 software versions of the GASNet Core API, enabling us to write distributed applications with software and hardware GASNet components transparently communicating with each other., Comment: Presented at First International Workshop on FPGAs for Software Programmers (FSP 2014) (arXiv:1408.4423)

Phonon densities of states (DOS) of bcc $\alpha$-$^{57}$Fe were measured from room temperature through the 1044K Curie transition and the 1185K fcc $\gamma$-Fe phase transition using nuclear resonant inelastic x-ray scattering. At higher temperatures all phonons shift to lower energies (soften) with thermal expansion, but the low transverse modes soften especially rapidly above 700K, showing strongly nonharmonic behavior that persists through the magnetic transition. Interatomic force constants for the bcc phase were obtained by iteratively fitting a Born-von K\'{a}rm\'{a}n model to the experimental phonon spectra using a genetic algorithm optimization. The second-nearest-neighbor fitted axial force constants weakened significantly at elevated temperatures. An unusually large nonharmonic behavior is reported, which increases the vibrational entropy and accounts for a contribution of 35 meV/atom in the free energy at high temperatures. The nonharmonic contribution to the vibrational entropy follows the thermal trend of the magnetic entropy, and may be coupled to magnetic excitations. A small change in vibrational entropy across the $\alpha$-$\gamma$ structural phase transformation is also reported.

The recent discovery of iron ferropnictide superconductors has received intensive concerns on magnetic involved superconductors. Prominent features of ferropnictide superconductors are becoming apparent: the parent compounds exhibit antiferromagnetic (AFM) ordered spin density wave (SDW) state; the magnetic phase transition is always accompanied to a crystal structural transition; superconductivity can be induced by suppressing the SDW phase via either chemical doping or applied external pressure to the parent state. These features generated considerable interests on the interplay between magnetism and structure in chemical doped samples, showing crystal structure transitions always precedes to or coincide with magnetic transition. Pressure tuned transition on the other hand would be more straightforward to superconducting mechanism studies since there are no disorder effects caused by chemical doping; however, remarkably little is known about the interplay in the parent compounds under controlled pressure due to the experimental challenge of in situ measuring both of magnetic & crystal structure evolution at high pressure & low temperatures. Here we show from combined synchrotron Mossbauer and x-ray diffraction at high pressures that the magnetic ordering surprisingly precedes the structural transition at high pressures in the parent compound BaFe2As2, in sharp contrast to the chemical doping case. The results can be well understood in terms of the spin fluctuations in the emerging nematic phase before the long range magnetic order that sheds new light on understanding how parent compound evolves from a SDW state to a superconducting phase, a key scientific inquiry of iron based superconductors., Comment: 18 pages, 4 figures

We report the first high pressure measurement of the Fe K-edge in hematite (Fe$_2$O$_3$) by X-ray absorption spectroscopy in partial fluorescence yield geometry. The pressure-induced evolution of the electronic structure as Fe$_2$O$_3$ transforms from a high-spin insulator to a low-spin metal is reflected in the x-ray absorption pre-edge. The crystal field splitting energy was found to increase monotonically with pressure up to 48 GPa, above which a series of phase transitions occur. Atomic multiplet, cluster diagonalization, and density-functional calculations were performed to simulate the pre-edge absorption spectra, showing good qualitative agreement with the measurements. The mechanism for the pressure-induced phase transitions of Fe$_2$O$_3$ is discussed and it is shown that ligand hybridization significantly reduces the critical high-spin/low-spin pressure., Comment: 5 pages, 4 figures and 1 table