Search

We present a rack-scale compute architecture for ML using multi-accelerator servers connected via chip-to-chip silicon photonic components. Our architecture achieves (1) multi-tenanted resource slicing without fragmentation, (2) 74% faster rack-scale collective communication, and (3) 1.7X speedup in end-to-end ML training throughput., Comment: Accepted at OFC 2025, https://www.ofcconference.org/en-us/home/program-speakers/symposia/advanced-packaging-and-integrated-optics/

Photonic integrated circuits are finding use in a variety of applications including optical transceivers, LIDAR, bio-sensing, photonic quantum computing, and Machine Learning (ML). In particular, with the exponentially increasing sizes of ML models, photonics-based accelerators are getting special attention as a sustainable solution because they can perform ML inferences with multiple orders of magnitude higher energy efficiency than CMOS-based accelerators. However, recent studies have shown that hardware manufacturing and infrastructure contribute significantly to the carbon footprint of computing devices, even surpassing the emissions generated during their use. For example, the manufacturing process accounts for 74% of the total carbon emissions from Apple in 2019. This prompts us to ask -- if we consider both the embodied (manufacturing) and operational carbon cost of photonics, is it indeed a viable avenue for a sustainable future? So, in this paper, we build a carbon footprint model for photonic chips and investigate the sustainability of photonics-based accelerators by conducting a case study on ADEPT, a photonics-based accelerator for deep neural network inference. Our analysis shows that photonics can reduce both operational and embodied carbon footprints with its high energy efficiency and at least 4$\times$ less fabrication carbon cost per unit area than 28 nm CMOS.

Over the past few years, silicon photonics-based computing has emerged as a promising alternative to CMOS-based computing for Deep Neural Networks (DNN). Unfortunately, the non-linear operations and the high-precision requirements of DNNs make it extremely challenging to design efficient silicon photonics-based systems for DNN inference and training. Hyperdimensional Computing (HDC) is an emerging, brain-inspired machine learning technique that enjoys several advantages over existing DNNs, including being lightweight, requiring low-precision operands, and being robust to noise introduced by the nonidealities in the hardware. For HDC, computing in-memory (CiM) approaches have been widely used, as CiM reduces the data transfer cost if the operands can fit into the memory. However, inefficient multi-bit operations, high write latency, and low endurance make CiM ill-suited for HDC. On the other hand, the existing electro-photonic DNN accelerators are inefficient for HDC because they are specifically optimized for matrix multiplication in DNNs and consume a lot of power with high-precision data converters. In this paper, we argue that photonic computing and HDC complement each other better than photonic computing and DNNs, or CiM and HDC. We propose PhotoHDC, the first-ever electro-photonic accelerator for HDC training and inference, supporting the basic, record-based, and graph encoding schemes. Evaluating with popular datasets, we show that our accelerator can achieve two to five orders of magnitude lower EDP than the state-of-the-art electro-photonic DNN accelerators for implementing HDC training and inference. PhotoHDC also achieves four orders of magnitude lower energy-delay product than CiM-based accelerators for both HDC training and inference.

Photonic computing is a compelling avenue for performing highly efficient matrix multiplication, a crucial operation in Deep Neural Networks (DNNs). While this method has shown great success in DNN inference, meeting the high precision demands of DNN training proves challenging due to the precision limitations imposed by costly data converters and the analog noise inherent in photonic hardware. This paper proposes Mirage, a photonic DNN training accelerator that overcomes the precision challenges in photonic hardware using the Residue Number System (RNS). RNS is a numeral system based on modular arithmetic, allowing us to perform high-precision operations via multiple low-precision modular operations. In this work, we present a novel micro-architecture and dataflow for an RNS-based photonic tensor core performing modular arithmetic in the analog domain. By combining RNS and photonics, Mirage provides high energy efficiency without compromising precision and can successfully train state-of-the-art DNNs achieving accuracy comparable to FP32 training. Our study shows that on average across several DNNs when compared to systolic arrays, Mirage achieves more than $23.8\times$ faster training and $32.1\times$ lower EDP in an iso-energy scenario and consumes $42.8\times$ lower power with comparable or better EDP in an iso-area scenario.

Photonic computing promises faster and more energy-efficient deep neural network (DNN) inference than traditional digital hardware. Advances in photonic computing can have profound impacts on applications such as autonomous driving and defect detection that depend on fast, accurate and energy efficient execution of image segmentation models. In this paper, we investigate image segmentation on photonic accelerators to explore: a) the types of image segmentation DNN architectures that are best suited for photonic accelerators, and b) the throughput and energy efficiency of executing the different image segmentation models on photonic accelerators, along with the trade-offs involved therein. Specifically, we demonstrate that certain segmentation models exhibit negligible loss in accuracy (compared to digital float32 models) when executed on photonic accelerators, and explore the empirical reasoning for their robustness. We also discuss techniques for recovering accuracy in the case of models that do not perform well. Further, we compare throughput (inferences-per-second) and energy consumption estimates for different image segmentation workloads on photonic accelerators. We discuss the challenges and potential optimizations that can help improve the application of photonic accelerators to such computer vision tasks.

Analog computing has reemerged as a promising avenue for accelerating deep neural networks (DNNs) due to its potential to overcome the energy efficiency and scalability challenges posed by traditional digital architectures. However, achieving high precision and DNN accuracy using such technologies is challenging, as high-precision data converters are costly and impractical. In this paper, we address this challenge by using the residue number system (RNS). RNS allows composing high-precision operations from multiple low-precision operations, thereby eliminating the information loss caused by the limited precision of the data converters. Our study demonstrates that analog accelerators utilizing the RNS-based approach can achieve ${\geq}99\%$ of FP32 accuracy for state-of-the-art DNN inference using data converters with only $6$-bit precision whereas a conventional analog core requires more than $8$-bit precision to achieve the same accuracy in the same DNNs. The reduced precision requirements imply that using RNS can reduce the energy consumption of analog accelerators by several orders of magnitude while maintaining the same throughput and precision. Our study extends this approach to DNN training, where we can efficiently train DNNs using $7$-bit integer arithmetic while achieving accuracy comparable to FP32 precision. Lastly, we present a fault-tolerant dataflow using redundant RNS error-correcting codes to protect the computation against noise and errors inherent within an analog accelerator.

The recent rise of large language models (LLMs) has resulted in increased efforts towards running LLMs at reduced precision. Running LLMs at lower precision supports resource constraints and furthers their democratization, enabling users to run billion-parameter LLMs on their personal devices. To supplement this ongoing effort, we propose INT-FP-QSim: an open-source simulator that enables flexible evaluation of LLMs and vision transformers at various numerical precisions and formats. INT-FP-QSim leverages existing open-source repositories such as TensorRT, QPytorch and AIMET for a combined simulator that supports various floating point and integer formats. With the help of our simulator, we survey the impact of different numerical formats on the performance of LLMs and vision transformers at 4-bit weights and 4-bit or 8-bit activations. We also compare recently proposed methods like Adaptive Block Floating Point, SmoothQuant, GPTQ and RPTQ on the model performances. We hope INT-FP-QSim will enable researchers to flexibly simulate models at various precisions to support further research in quantization of LLMs and vision transformers., Comment: This report is supplementary material to the open-source code available at: https://github.com/lightmatter-ai/INT-FP-QSim

Achieving high accuracy, while maintaining good energy efficiency, in analog DNN accelerators is challenging as high-precision data converters are expensive. In this paper, we overcome this challenge by using the residue number system (RNS) to compose high-precision operations from multiple low-precision operations. This enables us to eliminate the information loss caused by the limited precision of the ADCs. Our study shows that RNS can achieve 99% FP32 accuracy for state-of-the-art DNN inference using data converters with only $6$-bit precision. We propose using redundant RNS to achieve a fault-tolerant analog accelerator. In addition, we show that RNS can reduce the energy consumption of the data converters within an analog accelerator by several orders of magnitude compared to a regular fixed-point approach.

Existing methods to recover model accuracy on analog-digital hardware in the presence of quantization and analog noise include noise-injection training. However, it can be slow in practice, incurring high computational costs, even when starting from pretrained models. We introduce the Sensitivity-Aware Finetuning (SAFT) approach that identifies noise sensitive layers in a model, and uses the information to freeze specific layers for noise-injection training. Our results show that SAFT achieves comparable accuracy to noise-injection training and is 2x to 8x faster., Comment: 7 pages, 2 figures

Analog mixed-signal (AMS) devices promise faster, more energy-efficient deep neural network (DNN) inference than their digital counterparts. However, recent studies show that DNNs on AMS devices with fixed-point numbers can incur an accuracy penalty because of precision loss. To mitigate this penalty, we present a novel AMS-compatible adaptive block floating-point (ABFP) number representation. We also introduce amplification (or gain) as a method for increasing the accuracy of the number representation without increasing the bit precision of the output. We evaluate the effectiveness of ABFP on the DNNs in the MLPerf datacenter inference benchmark -- realizing less than $1\%$ loss in accuracy compared to FLOAT32. We also propose a novel method of finetuning for AMS devices, Differential Noise Finetuning (DNF), which samples device noise to speed up finetuning compared to conventional Quantization-Aware Training., Comment: 13 pages including Appendix, 7 figures, under submission at IEEE Transactions on Neural Networks and Learning Systems (TNNLS)

Advances in deep neural networks (DNNs) are transforming science and technology. However, the increasing computational demands of the most powerful DNNs limit deployment on low-power devices, such as smartphones and sensors -- and this trend is accelerated by the simultaneous move towards Internet-of-Things (IoT) devices. Numerous efforts are underway to lower power consumption, but a fundamental bottleneck remains due to energy consumption in matrix algebra, even for analog approaches including neuromorphic, analog memory and photonic meshes. Here we introduce and demonstrate a new approach that sharply reduces energy required for matrix algebra by doing away with weight memory access on edge devices, enabling orders of magnitude energy and latency reduction. At the core of our approach is a new concept that decentralizes the DNN for delocalized, optically accelerated matrix algebra on edge devices. Using a silicon photonic smart transceiver, we demonstrate experimentally that this scheme, termed Netcast, dramatically reduces energy consumption. We demonstrate operation in a photon-starved environment with 40 aJ/multiply of optical energy for 98.8% accurate image recognition and <1 photon/multiply using single photon detectors. Furthermore, we show realistic deployment of our system, classifying images with 3 THz of bandwidth over 86 km of deployed optical fiber in a Boston-area fiber network. Our approach enables computing on a new generation of edge devices with speeds comparable to modern digital electronics and power consumption that is orders of magnitude lower.

The number of parameters in deep neural networks (DNNs) is scaling at about 5$\times$ the rate of Moore's Law. To sustain this growth, photonic computing is a promising avenue, as it enables higher throughput in dominant general matrix-matrix multiplication (GEMM) operations in DNNs than their electrical counterpart. However, purely photonic systems face several challenges including lack of photonic memory and accumulation of noise. In this paper, we present an electro-photonic accelerator, ADEPT, which leverages a photonic computing unit for performing GEMM operations, a vectorized digital electronic ASIC for performing non-GEMM operations, and SRAM arrays for storing DNN parameters and activations. In contrast to prior works in photonic DNN accelerators, we adopt a system-level perspective and show that the gains while large are tempered relative to prior expectations. Our goal is to encourage architects to explore photonic technology in a more pragmatic way considering the system as a whole to understand its general applicability in accelerating today's DNNs. Our evaluation shows that ADEPT can provide, on average, 5.73$\times$ higher throughput per Watt compared to the traditional systolic arrays (SAs) in a full-system, and at least 6.8$\times$ and $2.5\times$ better throughput per Watt, compared to state-of-the-art electronic and photonic accelerators, respectively.

Reliable operation of photonic integrated circuits at cryogenic temperatures would enable new capabilities for emerging computing platforms, such as quantum technologies and low-power cryogenic computing. The silicon-on-insulator platform is a highly promising approach to developing large-scale photonic integrated circuits due to its exceptional manufacturability, CMOS compatibility and high component density. Fast, efficient and low-loss modulation at cryogenic temperatures in silicon, however, remains an outstanding challenge, particularly without the addition of exotic nonlinear optical materials. In this paper, we demonstrate DC-Kerr-effect-based modulation at a temperature of 5 K at GHz speeds, in a silicon photonic device fabricated exclusively within a CMOS process. This work opens up the path for the integration of DC Kerr modulators in large-scale photonic integrated circuits for emerging cryogenic classical and quantum computing applications.

Quantum key distribution (QKD) allows for secure communications safe against attacks by quantum computers. QKD protocols are performed by sending a sizeable, but finite, number of quantum signals between the distant parties involved. Many QKD experiments however predict their achievable key rates using asymptotic formulas, which assume the transmission of an infinite number of signals, partly because QKD proofs with finite transmissions (and finite key lengths) can be difficult. Here we develop a robust numerical approach for calculating the key rates for QKD protocols in the finite-key regime in terms of two novel semi-definite programs (SDPs). The first uses the relation between smooth min-entropy and quantum relative entropy, and the second uses the relation between the smooth min-entropy and quantum fidelity. We then solve these SDPs using convex optimization solvers and obtain some of the first numerical calculations of finite key rates for several different protocols, such as BB84, B92, and twin-field QKD. Our numerical approach democratizes the composable security proofs for QKD protocols where the derived keys can be used as an input to another cryptosystem.

Quantum algorithms for Noisy Intermediate-Scale Quantum (NISQ) machines have recently emerged as new promising routes towards demonstrating near-term quantum advantage (or supremacy) over classical systems. In these systems samples are typically drawn from probability distributions which --- under plausible complexity-theoretic conjectures --- cannot be efficiently generated classically. Rather than first define a physical system and then determine computational features of the output state, we ask the converse question: given direct access to the quantum state, what features of the generating system can we efficiently learn? In this work we introduce the Variational Quantum Unsampling (VQU) protocol, a nonlinear quantum neural network approach for verification and inference of near-term quantum circuits outputs. In our approach one can variationally train a quantum operation to unravel the action of an unknown unitary on a known input state; essentially learning the inverse of the black-box quantum dynamics. While the principle of our approach is platform independent, its implementation will depend on the unique architecture of a specific quantum processor. Here, we experimentally demonstrate the VQU protocol on a quantum photonic processor. Alongside quantum verification, our protocol has broad applications; including optimal quantum measurement and tomography, quantum sensing and imaging, and ansatz validation., Comment: Comments welcome. Updates references and acknowledgements

Photonic integrated circuits (PICs) provide a compact and stable platform for quantum photonics. Here we demonstrate a silicon photonics quantum key distribution (QKD) transmitter in the first high-speed polarization-based QKD field tests. The systems reach composable secret key rates of 950 kbps in a local test (on a 103.6-m fiber with a total emulated loss of 9.2 dB) and 106 kbps in an intercity metropolitan test (on a 43-km fiber with 16.4 dB loss). Our results represent the highest secret key generation rate for polarization-based QKD experiments at a standard telecom wavelength and demonstrate PICs as a promising, scalable resource for future formation of metropolitan quantum-secure communications networks.

Quantum key distribution (QKD) exploits the quantum nature of light to share provably secure keys, allowing secure communication in the presence of an eavesdropper. The first QKD schemes used photons encoded in two states, such as polarization. Recently, much effort has turned to large-alphabet QKD schemes, which encode photons in high-dimensional basis states. Compared to binary-encoded QKD, large-alphabet schemes can encode more secure information per detected photon, boosting secure communication rates, and also provide increased resilience to noise and loss. High-dimensional encoding may also improve the efficiency of other quantum information processing tasks, such as performing Bell tests and implementing quantum gates. Here, we demonstrate a large-alphabet QKD protocol based on high-dimensional temporal encoding. We achieve record secret-key rates and perform the first field demonstration of large-alphabet QKD. This demonstrates a new, practical way to optimize secret-key rates and marks an important step towards transmission of high-dimensional quantum states in deployed networks., Comment: 6 pages, 4 figures

Environmental noise and disorder play critical roles in quantum particle and wave transport in complex media, including solid-state and biological systems. Recent work has predicted that coupling between noisy environments and disordered systems, in which coherent transport has been arrested due to localization effects, could actually enhance transport. Photonic integrated circuits are promising platforms for studying such effects, with a central goal being the development of large systems providing low-loss, high-fidelity control over all parameters of the transport problem. Here, we fully map the role of disorder in quantum transport using a nanophotonic processor consisting of a mesh of 88 generalized beamsplitters programmable on microsecond timescales. Over 64,400 transport experiments, we observe several distinct transport regimes, including environment-assisted quantum transport and the ''quantum Goldilocks'' regime in strong, statically disordered discrete-time systems. Low loss and high-fidelity programmable transformations make this nanophotonic processor a promising platform for many-boson quantum simulation experiments., Comment: 4 figures, 8 pages

High-dimensional quantum key distribution (HD-QKD) allows two parties to generate multiple secure bits of information per detected photon. In this work, we show that decoy state protocols can be practically implemented for HD-QKD using only one or two decoy states. HD-QKD with two decoy states, under realistic experimental constraints, can generate multiple secure bits per coincidence at distances over 200 km and at rates similar to those achieved by a protocol with infinite decoy states. Furthermore, HD-QKD with only one decoy state is practical at short distances, where it is almost as secure as a protocol with two decoy states. HD-QKD with only one or two decoy states can therefore be implemented to optimize the rate of secure quantum communications., Comment: 11 pages, 3 figures

We present a method of calculating the strong-field gravitational lensing caused by many analytic and numerical spacetimes. We use this procedure to calculate the distortion caused by isolated black holes and by numerically evolved black hole binaries. We produce both demonstrative images illustrating details of the spatial distortion and realistic images of collections of stars taking both lensing amplification and redshift into account. On large scales the lensing from inspiraling binaries resembles that of single black holes, but on small scales the resulting images show complex and in some cases self-similar structure across different angular scales., Comment: 10 pages, 12 figures. Supplementary images and movies can be found at http://www.black-holes.org/the-science-numerical-relativity/numerical-relativity/gravitational-lensing

A relativistic deep space positioning system has been proposed using four or more pulsars with stable repetition rates. (Each pulsar emits pulses at a fixed repetition period in its rest frame.) The positioning system uses the fact that an event in spacetime can be fully described by emission coordinates: the proper emission time of each pulse measured at the event. The proper emission time of each pulse from four different pulsars---interpolated as necessary---provides the four spacetime coordinates of the reception event in the emission coordinate system. If more than four pulsars are available, the redundancy can improve the accuracy of the determination and/or resolve degeneracies resulting from special geometrical arrangements of the sources and the event. We introduce a robust numerical approach to measure the emission coordinates of an event in any arbitrary spacetime geometry. Our approach uses a continuous solution of the eikonal equation describing the backward null cone from the event. The pulsar proper time at the instant the null cone intersects the pulsar world line is one of the four required coordinates. The process is complete (modulo degeneracies) when four pulsar world lines have been crossed by the light cone. The numerical method is applied in two different examples: measuring emission coordinates of an event in Minkowski spacetime using pulses from four pulsars stationary in the spacetime; and measuring emission coordinates of an event in Schwarzschild spacetime using pulses from four pulsars freely falling toward a static black hole. These numerical simulations are merely exploratory, but with improved resolution and computational resources the method can be applied to more pertinent problems. For instance one could measure the emission coordinates, and therefore the trajectory, of the Earth., Comment: 9 pages, 2 figures, v3: replaced with version accepted by Phys. Rev. D