Search

Physical neural networks (PNNs) are a class of neural-like networks that leverage the properties of physical systems to perform computation. While PNNs are so far a niche research area with small-scale laboratory demonstrations, they are arguably one of the most underappreciated important opportunities in modern AI. Could we train AI models 1000x larger than current ones? Could we do this and also have them perform inference locally and privately on edge devices, such as smartphones or sensors? Research over the past few years has shown that the answer to all these questions is likely "yes, with enough research": PNNs could one day radically change what is possible and practical for AI systems. To do this will however require rethinking both how AI models work, and how they are trained - primarily by considering the problems through the constraints of the underlying hardware physics. To train PNNs at large scale, many methods including backpropagation-based and backpropagation-free approaches are now being explored. These methods have various trade-offs, and so far no method has been shown to scale to the same scale and performance as the backpropagation algorithm widely used in deep learning today. However, this is rapidly changing, and a diverse ecosystem of training techniques provides clues for how PNNs may one day be utilized to create both more efficient realizations of current-scale AI models, and to enable unprecedented-scale models., Comment: 29 pages, 4 figures

On-chip photonic processors for neural networks have potential benefits in both speed and energy efficiency but have not yet reached the scale at which they can outperform electronic processors. The dominant paradigm for designing on-chip photonics is to make networks of relatively bulky discrete components connected by one-dimensional waveguides. A far more compact alternative is to avoid explicitly defining any components and instead sculpt the continuous substrate of the photonic processor to directly perform the computation using waves freely propagating in two dimensions. We propose and demonstrate a device whose refractive index as a function of space, $n(x,z)$, can be rapidly reprogrammed, allowing arbitrary control over the wave propagation in the device. Our device, a 2D-programmable waveguide, combines photoconductive gain with the electro-optic effect to achieve massively parallel modulation of the refractive index of a slab waveguide, with an index modulation depth of $10^{-3}$ and approximately $10^4$ programmable degrees of freedom. We used a prototype device with a functional area of $12\,\text{mm}^2$ to perform neural-network inference with up to 49-dimensional input vectors in a single pass, achieving 96% accuracy on vowel classification and 86% accuracy on $7 \times 7$-pixel MNIST handwritten-digit classification. This is a scale beyond that of previous photonic chips relying on discrete components, illustrating the benefit of the continuous-waves paradigm. In principle, with large enough chip area, the reprogrammability of the device's refractive index distribution enables the reconfigurable realization of any passive, linear photonic circuit or device. This promises the development of more compact and versatile photonic systems for a wide range of applications, including optical processing, smart sensing, spectroscopy, and optical communications.

Multimode squeezed states of light have been proposed as a resource for achieving quantum advantage in computing and sensing. Recent experiments that demonstrate multimode Gaussian states to this end have most commonly opted for spatial or temporal modes, whereas a complete system based on frequency modes has yet to be realized. Instead, we show how to use the frequency modes simultaneously squeezed in a conventional, single-spatial-mode, optical parametric amplifier when pumped by ultrashort pulses. Specifically, we show how adiabatic frequency conversion can be used not only to convert the quantum state from infrared to visible wavelengths, but to concurrently manipulate the joint spectrum. This near unity-efficiency quantum frequency conversion, over a bandwidth >45 THz and, to our knowledge, the broadest to date, allows us to measure the state with an electron-multiplying CCD (EMCCD) camera-based spectrometer, at non-cryogenic temperatures. We demonstrate the squeezing of >400 frequency modes, with a mean of approximately 700 visible photons per shot. Our work shows how many-mode quantum states of light can be generated, manipulated, and measured with efficient use of hardware resources -- in our case, using one pulsed laser, two nonlinear crystals, and one camera. This ability to produce, with modest hardware resources, large multimode squeezed states with partial programmability motivates the use of frequency encoding for photonics-based quantum information processing.

Quantum reservoir computing (QRC) has been proposed as a paradigm for performing machine learning with quantum processors where the training is efficient in the number of required runs of the quantum processor and takes place in the classical domain, avoiding the issue of barren plateaus in parameterized-circuit quantum neural networks. It is natural to consider using a quantum processor based on superconducting circuits to classify microwave signals that are analog -- continuous in time. However, while theoretical proposals of analog QRC exist, to date QRC has been implemented using circuit-model quantum systems -- imposing a discretization of the incoming signal in time, with each time point input by executing a gate operation. In this paper we show how a quantum superconducting circuit comprising an oscillator coupled to a qubit can be used as an analog quantum reservoir for a variety of classification tasks, achieving high accuracy on all of them. Our quantum system was operated without artificially discretizing the input data, directly taking in microwave signals. Our work does not attempt to address the question of whether QRCs could provide a quantum computational advantage in classifying pre-recorded classical signals. However, beyond illustrating that sophisticated tasks can be performed with a modest-size quantum system and inexpensive training, our work opens up the possibility of achieving a different kind of advantage than a purely computational advantage: superconducting circuits can act as extremely sensitive detectors of microwave photons; our work demonstrates processing of ultra-low-power microwave signals in our superconducting circuit, and by combining sensitive detection with QRC processing within the same system, one could achieve a quantum sensing-computational advantage, i.e., an advantage in the overall analysis of microwave signals comprising just a few photons.

Over the last few decades, nonlinear optics has become significantly more nonlinear, traversing nearly a billionfold improvement in energy efficiency, with ultrafast nonlinear nanophotonics in particular emerging as a frontier for combining both spatial and temporal engineering. At present, cutting-edge experiments in nonlinear nanophotonics place us just above the mesoscopic regime, where a few hundred photons suffice to trigger nonlinear saturation. In contrast to classical or deep-quantum optics, the mesoscale is characterized by dynamical interactions between mean-field, Gaussian, and non-Gaussian quantum features, all within a close hierarchy of scales. When combined with the inherent multimode complexity of optical fields, such hybrid quantum-classical dynamics present theoretical, experimental, and engineering challenges to the contemporary framework of quantum optics. In this review, we highlight the unique physics that emerges in multimode nonlinear optics at the mesoscale and outline key principles for exploiting both classical and quantum features to engineer novel functionalities. We briefly survey the experimental landscape and draw attention to outstanding technical challenges in materials, dispersion engineering, and device design for accessing mesoscopic operation. Finally, we speculate on how these capabilities might usher in some new paradigms in quantum photonics, from quantum-augmented information processing to nonclassical-light-driven dynamics and phenomena to all-optical non-Gaussian measurement and sensing. The physics unlocked at the mesoscale present significant challenges and opportunities in theory and experiment alike, and this review is intended to serve as a guidepost as we begin to navigate this new frontier in ultrafast quantum nonlinear optics., Comment: The first two authors contributed equally to this work; 26 pages, 7 figures

In broadband quantum optical systems, nonlinear interactions among a large number of frequency components induce complex dynamics that may defy heuristic analysis. In this work we introduce a perturbative framework for factoring out reservoir degrees of freedom and establishing a concise effective model (effective field theory) for the remaining system. Our approach combines approximate diagonalization of judiciously partitioned subsystems with master equation techniques. We consider cascaded optical $\chi^{(2)}$ (quadratic) nonlinearities as an example and show that the dynamics can be construed (to leading order) as self-phase modulations of dressed fundamental modes plus cross-phase modulations of dressed fundamental and second-harmonic modes. We then formally eliminate the second-harmonic degrees of freedom and identify emergent features of the fundamental wave dynamics, such as two-photon loss channels, and examine conditions for accuracy of the reduced model in dispersive and dissipative parameter regimes. Our results highlight the utility of system-reservoir methods for deriving accurate, intuitive reduced models for complex dynamics in broadband nonlinear quantum photonics., Comment: 25 pages, 8 figures

Optical imaging is commonly used for both scientific and technological applications across industry and academia. In image sensing, a measurement, such as of an object's position, is performed by computational analysis of a digitized image. An emerging image-sensing paradigm breaks this delineation between data collection and analysis by designing optical components to perform not imaging, but encoding. By optically encoding images into a compressed, low-dimensional latent space suitable for efficient post-analysis, these image sensors can operate with fewer pixels and fewer photons, allowing higher-throughput, lower-latency operation. Optical neural networks (ONNs) offer a platform for processing data in the analog, optical domain. ONN-based sensors have however been limited to linear processing, but nonlinearity is a prerequisite for depth, and multilayer NNs significantly outperform shallow NNs on many tasks. Here, we realize a multilayer ONN pre-processor for image sensing, using a commercial image intensifier as a parallel optoelectronic, optical-to-optical nonlinear activation function. We demonstrate that the nonlinear ONN pre-processor can achieve compression ratios of up to 800:1 while still enabling high accuracy across several representative computer-vision tasks, including machine-vision benchmarks, flow-cytometry image classification, and identification of objects in real scenes. In all cases we find that the ONN's nonlinearity and depth allowed it to outperform a purely linear ONN encoder. Although our experiments are specialized to ONN sensors for incoherent-light images, alternative ONN platforms should facilitate a range of ONN sensors. These ONN sensors may surpass conventional sensors by pre-processing optical information in spatial, temporal, and/or spectral dimensions, potentially with coherent and quantum qualities, all natively in the optical domain.

We study the emergence of non-Gaussian quantum features in pulsed squeezed light generation with a mesoscopic number (i.e., dozens to hundreds) of pump photons. Due to the strong optical nonlinearities necessarily involved in this regime, squeezing occurs alongside significant pump depletion, compromising the predictions made by conventional semiclassical models for squeezing. Furthermore, nonlinear interactions among multiple frequency modes render the system dynamics exponentially intractable in na\"ive quantum models, requiring a more sophisticated modeling framework. To this end, we construct a nonlinear Gaussian approximation to the squeezing dynamics, defining a "Gaussian interaction frame" (GIF) in which non-Gaussian quantum dynamics can be isolated and concisely described using a few dominant (i.e., principal) supermodes. Numerical simulations of our model reveal non-Gaussian distortions of squeezing in the mesoscopic regime, largely associated with signal-pump entanglement. We argue that the state of the art in nonlinear nanophotonics is quickly approaching this regime, providing an all-optical platform for experimental studies of the semiclassical-to-quantum transition in a rich paradigm of coherent, multimode nonlinear dynamics. Mesoscopic pulsed squeezing thus provides an intriguing case study of the rapid rise in dynamic complexity associated with semiclassical-to-quantum crossover, which we view as a correlate of the emergence of new information-processing capacities in the quantum regime., Comment: The first two authors contributed equally to this work; 16 pages, 7 figures

We study the performance scaling of three quantum algorithms for combinatorial optimization: measurement-feedback coherent Ising machines (MFB-CIM), discrete adiabatic quantum computation (DAQC), and the D\"urr-Hoyer algorithm for quantum minimum finding (DH-QMF) that is based on Grover's search. We use MaxCut problems as our reference for comparison, and time-to-solution (TTS) as a practical measure of performance for these optimization algorithms. We empirically observe a $\Theta(2^{\sqrt{n}})$ scaling for the median TTS for MFB-CIM, in comparison to the exponential scaling with the exponent $n$ for DAQC and the provable $\widetilde{\mathcal O}\left(\sqrt{2^n}\right)$ scaling for DH-QMF. We conclude that these scaling complexities result in a dramatic performance advantage for MFB-CIM in comparison to the other two algorithms for solving MaxCut problems., Comment: 24 pages, 20 figures

Deep learning has rapidly become a widespread tool in both scientific and commercial endeavors. Milestones of deep learning exceeding human performance have been achieved for a growing number of tasks over the past several years, across areas as diverse as game-playing, natural-language translation, and medical-image analysis. However, continued progress is increasingly hampered by the high energy costs associated with training and running deep neural networks on electronic processors. Optical neural networks have attracted attention as an alternative physical platform for deep learning, as it has been theoretically predicted that they can fundamentally achieve higher energy efficiency than neural networks deployed on conventional digital computers. Here, we experimentally demonstrate an optical neural network achieving 99% accuracy on handwritten-digit classification using ~3.2 detected photons per weight multiplication and ~90% accuracy using ~0.64 photons (~$2.4 \times 10^{-19}$ J of optical energy) per weight multiplication. This performance was achieved using a custom free-space optical processor that executes matrix-vector multiplications in a massively parallel fashion, with up to ~0.5 million scalar (weight) multiplications performed at the same time. Using commercially available optical components and standard neural-network training methods, we demonstrated that optical neural networks can operate near the standard quantum limit with extremely low optical powers and still achieve high accuracy. Our results provide a proof-of-principle for low-optical-power operation, and with careful system design including the surrounding electronics used for data storage and control, open up a path to realizing optical processors that require only $10^{-16}$ J total energy per scalar multiplication -- which is orders of magnitude more efficient than current digital processors., Comment: 42 pages, 21 figures

Deep neural networks have become a pervasive tool in science and engineering. However, modern deep neural networks' growing energy requirements now increasingly limit their scaling and broader use. We propose a radical alternative for implementing deep neural network models: Physical Neural Networks. We introduce a hybrid physical-digital algorithm called Physics-Aware Training to efficiently train sequences of controllable physical systems to act as deep neural networks. This method automatically trains the functionality of any sequence of real physical systems, directly, using backpropagation, the same technique used for modern deep neural networks. To illustrate their generality, we demonstrate physical neural networks with three diverse physical systems-optical, mechanical, and electrical. Physical neural networks may facilitate unconventional machine learning hardware that is orders of magnitude faster and more energy efficient than conventional electronic processors.

We show that the nonlinear stochastic dynamics of a measurement-feedback-based coherent Ising machine (MFB-CIM) in the presence of quantum noise can be exploited to sample degenerate ground and low-energy spin configurations of the Ising model. We formulate a general discrete-time Gaussian-state model of the MFB-CIM which faithfully captures the nonlinear dynamics present at and above system threshold. This model overcomes the limitations of both mean-field models, which neglect quantum noise, and continuous-time models, which assume long photon lifetimes. Numerical simulations of our model show that when the MFB-CIM is operated in a quantum-noise-dominated regime with short photon lifetimes (i.e., low cavity finesse), homodyne monitoring of the system can efficiently produce samples of low-energy Ising spin configurations, requiring many fewer roundtrips to sample than suggested by established high-finesse, continuous-time models. We find that sampling performance is robust to, or even improved by, turning off or altogether reversing the sign of the parametric drive, but performance is critically reduced in the absence of optical nonlinearity. For the class of MAX-CUT problems with binary-signed edge weights, the number of roundtrips sufficient to fully sample all spin configurations up to the first-excited Ising energy, including all degeneracies, scales as $1.08^N$. At a problem size of $N = 100$ with a few dozen (median of 20) such desired configurations per instance, we have found median sufficient sampling times of $6\times10^6$ roundtrips; in an experimental implementation of an MFB-CIM with a 10 GHz repetition rate, this corresponds to a wall-clock sampling time of 60 ms., Comment: The first two authors contributed equally to this work. 24 pages, 9 figures

The advent of dispersion-engineered and highly nonlinear nanophotonics is expected to open up an all-optical path towards the strong-interaction regime of quantum optics by combining high transverse field confinement with ultra-short-pulse operation. Obtaining a full understanding of photon dynamics in such broadband devices, however, poses major challenges in the modeling and simulation of multimode non-Gaussian quantum physics, highlighting the need for sophisticated reduced models that facilitate efficient numerical study while providing useful physical insight. In this manuscript, we review our recent efforts in modeling broadband optical systems at varying levels of abstraction and generality, ranging from multimode extensions of quantum input-output theory for sync-pumped oscillators to the development of numerical methods based on a field-theoretic description of nonlinear waveguides. We expect our work not only to guide ongoing theoretical and experimental efforts towards next-generation quantum devices but also to uncover essential physics of broadband quantum photonics., Comment: The first two authors contributed equally to this work; 11 pages, 2 figures

Ultra-short pulses propagating in nonlinear nanophotonic waveguides can simultaneously leverage both temporal and spatial field confinement, promising a route towards single-photon nonlinearities in an all-photonic platform. In this multimode quantum regime, however, faithful numerical simulations of pulse dynamics na\"ively require a representation of the state in an exponentially large Hilbert space. Here, we employ a time-domain, matrix product state (MPS) representation to enable efficient simulations by exploiting the entanglement structure of the system. In order to extract physical insight from these simulations, we develop an algorithm to unravel the MPS quantum state into constituent temporal supermodes, enabling, e.g., access to the phase-space portraits of arbitrary pulse waveforms. As a demonstration, we perform exact numerical simulations of a Kerr soliton in the quantum regime. We observe the development of non-classical Wigner-function negativity in the solitonic mode as well as quantum corrections to the semiclassical dynamics of the pulse. A similar analysis of $\chi^{(2)}$ simultons reveals a unique entanglement structure between the fundamental and second harmonic. Our approach is also readily compatible with quantum trajectory theory, allowing full quantum treatment of propagation loss and decoherence. We expect this work to establish the MPS technique as part of a unified engineering framework for the emerging field of broadband quantum photonics., Comment: 12 pages, 7 figures

We introduce a theoretical framework based on Fano's theory of discrete-continuum interactions to analyze the quantum dynamics of broadband parametric downconversion (PDC) in the few-pump-photon regime of nonlinear quantum nanophotonics. Applying this unified analytic approach to 1D $\chi^{(2)}$-nonlinear waveguides, we find a host of remarkable dynamical features due to the coupling of a discrete pump state to the signal continuum, from unit-efficiency (i.e., complete) downconversion when the coupling is dissipative, to Rabi-like oscillations with sub-exponential decay when it is dispersive. The theory provides a straightforward way to analytically compute a full characterization of the PDC dynamics, including the complete eigensystem of the continuum Hamiltonian and expressions for the signal biphoton correlation function. We also apply the theory to study a pair of linearly coupled $\chi^{(2)}$ waveguides, where two discrete pump states simultaneously downconvert into a common-mode signal continuum, resulting in Fano interference that critically affects the PDC rate. Under appropriate conditions, the theory predicts characteristic Fano lineshapes and even complete destructive interference resulting in the full suppression of PDC, due to the formation of a bound pump state in the continuum. Generalizing further, we show that the framework can also be applied to higher-order parametric processes such as parametric three-photon generation, and we also find numerical signatures that Fano-type interactions occur even for multi-photon PDC under stronger pumping. Our results establish broadband PDC as yet another physical system natively exhibiting Fano-type interactions and advance a theoretical framework in which to understand the complicated quantum dynamics of strongly nonlinear broadband quantum optics., Comment: The first two authors contributed equally to this work; 22 pages, 10 figures

We propose a deterministic, measurement-free implementation of a cubic phase gate for continuous-variable quantum information processing. In our scheme, the applications of displacement and squeezing operations allow us to engineer the effective evolution of the quantum state propagating through an optical Kerr nonlinearity. Under appropriate conditions, we show that the input state evolves according to a cubic phase Hamiltonian, and we find that the cubic phase gate error decreases inverse-quartically with the amount of quadrature squeezing, even in the presence of linear loss. We also show how our scheme can be adapted to deterministically generate a nonclassical approximate cubic phase state with high fidelity using a ratio of native nonlinearity to linear loss of only $10^{-4}$, indicating that our approach may be experimentally viable in the near term even on all-optical platforms, e.g., using quantum solitons in pulsed nonlinear nanophotonics., Comment: 10 pages, 7 figures. The first two authors contributed equally to this work

Quantum annealing is a promising approach to heuristically solving difficult combinatorial optimization problems. However, the connectivity limitations in current devices lead to an exponential degradation of performance on general problems. We propose an architecture for a quantum annealer that achieves full connectivity and full programmability while using a number of physical resources only linear in the number of spins. We do so by application of carefully engineered periodic modulations of oscillator-based qubits, resulting in a Floquet Hamiltonian in which all the interactions are tunable; this flexibility comes at a cost of the coupling strengths between spins being smaller than they would be had the spins been directly coupled. Our proposal is well-suited to implementation with superconducting circuits, and we give analytical and numerical evidence that fully connected, fully programmable quantum annealers with $1000$ qubits could be constructed with Josephson parametric oscillators having cavity-photon lifetimes of $\sim 100$ microseconds, and other system-parameter values that are routinely achieved with current technology. Our approach could also have impact beyond quantum annealing, since it readily extends to bosonic quantum simulators and would allow the study of models with arbitrary connectivity between lattice sites., Comment: The first two authors contributed equally to this work. 13 pages, 5 figures (main text); 54 pages, 32 figures (supplementary)

We present a new scheme to efficiently establish entanglement between optical modes in a time-multiplexed coherent Ising machine (CIM) by means of nonlocal measurement and feedback. We numerically simulate and evaluate the generation of steady-state entanglement in a system with nearest-neighbor interactions on a 1D ring, and we compare the results to those of a conventional CIM with all-optical interactions mediated by delay lines. We show that the delay-line architecture has a fundamental limit on accessible entanglement that our measurement-based scheme can substantially surpass. These results motivate the study of non-classical correlations in CIMs with levels of entanglement previously considered to be experimentally impractical., Comment: 5 pages, 5 figures (main text); 6 pages, 1 figure (supplementary)

We propose a theoretical scheme to deterministically generate Fock states in a Kerr cavity thorough adiabatic variation of the driving field strength and the cavity detuning. We show that the required time to generate a $n$-photon Fock state scales as the square root of $n$. Requirements for the Kerr coefficient relative to the system decoherence rate are provided as a function of desired state fidelity, indicating that the scheme is potentially realizable with the present state of the art in microwave superconducting circuits.

The quantum features of ultrashort-pulse optical parametric oscillators (OPOs) are investigated theoretically in the nonlinear regime near and above threshold. Viewing the pulsed OPO as a multimode open quantum system, we rigorously derive a general input-output model that features nonlinear coupling among many cavity (i.e., system) signal modes and a broadband single-pass (i.e., reservoir) pump field. Under appropriate assumptions, our model produces a Lindblad master equation with multimode nonlinear Lindblad operators describing two-photon dissipation and a multimode four-wave-mixing Hamiltonian describing a broadband, dispersive optical cascade, which we show is required to preserve causality. To simplify the multimode complexity of the model, we employ a supermode decomposition to perform numerical simulations in the regime where the pulsed supermodes experience strong single-photon nonlinearity. We find that the quantum nonlinear dynamics induces pump depletion as well as corrections to the below-threshold squeezing spectrum predicted by linearized models. We also observe the formation of non-Gaussian states with Wigner-function negativity and show that the multimode interactions with the pump, both dissipative and dispersive, can act as effective decoherence channels. Finally, we briefly discuss some experimental considerations for potentially observing such quantum nonlinear phenomena with ultrashort-pulse OPOs on nonlinear nanophotonic platforms., Comment: The first three authors contributed equally to this work. 26 pages, 9 figures

Physical annealing systems provide heuristic approaches to solving NP-hard Ising optimization problems. Here, we study the performance of two types of annealing machines--a commercially available quantum annealer built by D-Wave Systems, and measurement-feedback coherent Ising machines (CIMs) based on optical parametric oscillator networks--on two classes of problems, the Sherrington-Kirkpatrick (SK) model and MAX-CUT. The D-Wave quantum annealer outperforms the CIMs on MAX-CUT on regular graphs of degree 3. On denser problems, however, we observe an exponential penalty for the quantum annealer ($\exp(-\alpha_\textrm{DW} N^2)$) relative to CIMs ($\exp(-\alpha_\textrm{CIM} N)$) for fixed anneal times, on both the SK model and on 50%-edge-density MAX-CUT, where the coefficients $\alpha_\textrm{CIM}$ and $\alpha_\textrm{DW}$ are problem-class-dependent. On instances with over $50$ vertices, a several-orders-of-magnitude time-to-solution difference exists between CIMs and the D-Wave annealer. An optimal-annealing-time analysis is also consistent with a significant projected performance difference. The difference in performance between the sparsely connected D-Wave machine and the measurement-feedback facilitated all-to-all connectivity of the CIMs provides strong experimental support for efforts to increase the connectivity of quantum annealers., Comment: 12 pages, 5 figures, 1 table (main text); 14 pages, 12 figures, 2 tables (supplementary)