Search

Deep neural networks (DNNs) have achieved exceptional performance across various fields by learning complex nonlinear mappings from large-scale datasets. However, they encounter challenges such as high computational costs and limited interpretability. To address these issues, hybrid approaches that integrate physics with AI are gaining interest. This paper introduces a novel physics-based AI model called the "Nonlinear Schr\"odinger Network", which treats the Nonlinear Schr\"odinger Equation (NLSE) as a general-purpose trainable model for learning complex patterns including nonlinear mappings and memory effects from data. Existing physics-informed machine learning methods use neural networks to approximate the solutions of partial differential equations (PDEs). In contrast, our approach directly treats the PDE as a trainable model to obtain general nonlinear mappings that would otherwise require neural networks. As a type of physics-AI symbiosis, it offers a more interpretable and parameter-efficient alternative to traditional black-box neural networks, achieving comparable or better accuracy in some time series classification tasks while significantly reducing the number of required parameters. Notably, the trained Nonlinear Schr\"odinger Network is interpretable, with all parameters having physical meanings as properties of a virtual physical system that transforms the data to a more separable space. This interpretability allows for insight into the underlying dynamics of the data transformation process. Applications to time series forecasting have also been explored. While our current implementation utilizes the NLSE, the proposed method of using physics equations as trainable models to learn nonlinear mappings from data is not limited to the NLSE and may be extended to other master equations of physics.

A single-shot measurement technique for ultrafast phenomena with high throughput enables the capture of rare events within a short time scale, facilitating the exploration of rare ultrafast processes. Photonic time stretch stands out as a highly effective method for both detecting rapid events and achieving remarkable speed in imaging and ranging applications. The current time stretch method relies on costly passive mode-locked lasers with continuous and fixed spectra to capture fast transients and dilate their time scale using dispersion. This hinders the broad application of time stretch technology and presents synchronization challenges with ultrafast events for measurement. Here we report the first implementation of time stretch using continuous wave (CW) diode lasers with discrete and tunable spectra that are common in WDM optical communication. This approach offers the potential for more cost-effective and compact time stretch systems and simplifies laser synchronization with the input signal. Two different embodiments in the United States and Japan demonstrate the technique's operation and limitations, and potential applications to time stretch imaging and angular light scattering.

Time stretch instruments have been exceptionally successful in discovering single-shot ultrafast phenomena such as optical rogue waves and have led to record-speed microscopy, spectroscopy, lidar, etc. These instruments encode the ultrafast events into the spectrum of a femtosecond pulse and then dilate the time scale of the data using group velocity dispersion. Generating as much as Tbit per second of data, they are ideal partners for deep learning networks which by their inherent complexity, require large datasets for training. However, the inference time scale of neural networks in the millisecond regime is orders of magnitude longer than the data acquisition rate of time stretch instruments. This underscores the need to explore means where some of the lower-level computational tasks can be done while the data is still in the optical domain. The Nonlinear Schr\"{o}dinger Kernel computing addresses this predicament. It utilizes optical nonlinearities to map the data onto a new domain in which classification accuracy is enhanced, without increasing the data dimensions. One limitation of this technique is the fixed optical transfer function, which prevents training and generalizability. Here we show that the optical kernel can be effectively tuned and trained by utilizing digital phase encoding of the femtosecond laser pulse leading to a reduction of the error rate in data classification.

PhyCV is the first computer vision library which utilizes algorithms directly derived from the equations of physics governing physical phenomena. The algorithms appearing in the current release emulate, in a metaphoric sense, the propagation of light through a physical medium with natural and engineered diffractive properties followed by coherent detection. Unlike traditional algorithms that are a sequence of hand-crafted empirical rules or deep learning algorithms that are usually data-driven and computationally heavy, physics-inspired algorithms leverage physical laws of nature as blueprints for inventing algorithms. PhyCV features low-dimensionality and high- efficiency, making it ideal for edge computing applications. We demonstrate real-time video processing on NVIDIA Jetson Nano using PhyCV. In addition, these algorithms have the potential to be implemented in real physical devices for fast and efficient computation in the form of analog computing. The open-sourced code is available at https://github.com/JalaliLabUCLA/phycv

The history of computing started with analog computers consisting of physical devices performing specialized functions such as predicting the trajectory of cannon balls. In modern times, this idea has been extended, for example, to ultrafast nonlinear optics serving as a surrogate analog computer to probe the behavior of complex phenomena such as rogue waves. Here we discuss a new paradigm where physical phenomena coded as an algorithm perform computational imaging tasks. Specifically, diffraction followed by coherent detection, not in its analog realization but when coded as an algorithm, becomes an image enhancement tool. Vision Enhancement via Virtual diffraction and coherent Detection (VEViD) introduced here reimagines a digital image as a spatially varying metaphoric light field and then subjects the field to the physical processes akin to diffraction and coherent detection. The term "Virtual" captures the deviation from the physical world. The light field is pixelated and the propagation imparts a phase with an arbitrary dependence on frequency which can be different from the quadratic behavior of physical diffraction. Temporal frequencies exist in three bands corresponding to the RGB color channels of a digital image. The phase of the output, not the intensity, represents the output image. VEViD is a high-performance low-light-level and color enhancement tool that emerges from this paradigm. The algorithm is interpretable and computationally efficient. We demonstrate image enhancement of 4k video at 200frames per second and show the utility of this physical algorithm in improving the accuracy of object detection by neural networks without having to retrain model for low-light conditions. The application of VEViD to color enhancement is also demonstrated.

Phase-Stretch Adaptive Gradient-Field Extractor (PAGE) is an edge detection algorithm that is inspired by physics of electromagnetic diffraction and dispersion. A computational imaging algorithm, it identifies edges, their orientations and sharpness in a digital image where the image brightness changes abruptly. Edge detection is a basic operation performed by the eye and is crucial to visual perception. PAGE embeds an original image into a set of feature maps that can be used for object representation and classification. The algorithm performs exceptionally well as an edge and texture extractor in low light level and low contrast images. This manuscript is prepared to support the open-source code which is being simultaneously made available within the GitHub repository https://github.com/JalaliLabUCLA/Phase-Stretch-Adaptive-Gradient-field-Extractor/.

The phenomenal success of physics in explaining nature and designing hardware is predicated on efficient computational models. A universal codebook of physical laws defines the computational rules and a physical system is an interacting ensemble governed by these rules. Led by deep neural networks, artificial intelligence (AI) has introduced an alternate end-to-end data-driven computational framework, with astonishing performance gains in image classification and speech recognition and fueling hopes for a novel approach to discovering physics itself. These gains, however, come at the expense of interpretability and also computational efficiency; a trend that is on a collision course with the expected end of semiconductor scaling known as the Moore's Law. With focus on photonic applications, this paper argues how an emerging symbiosis of physics and artificial intelligence can overcome such formidable challenges, thereby not only extending the latter's spectacular rise but also transforming the direction of physical science.

Alternative machine learning approaches that are computationally light with low latency and can work with only a small training dataset are needed for applications where the insatiable demand of deep learning methods for computing power and large training data cannot be met. We show that spectral mapping of data onto femtosecond optical pulses and a projection into an implicit, higher dimensional space using nonlinear optical dynamics increases the accuracy and reduces the latency in data classification by several orders of magnitude. The approach is validated by the classification of various datasets, including brain intracranial pressure, cancer cell imaging, spoken digit recognition, and the classic exclusive OR benchmark for nonlinear classification. The concept is demonstrated by seeding the nonlinear dynamics that are responsible for many fascinating natural phenomena, such as optical rogue waves, with the data before processing the output with a light classifier. A quantitative comparison with a well-known numerical technique is used to provide insight into this physical technique. Single-shot operation is demonstrated using time stretch data acquisition., Comment: Updated the manuscript, include more results

With the advent of the 5G wireless networks, achieving tens of gigabits per second throughputs and low, milliseconds, latency has become a reality. This level of performance will fuel numerous real-time applications, such as autonomy and augmented reality, where the computationally heavy tasks can be performed in the cloud. The increase in the bandwidth along with the use of dense constellations places a significant burden on the speed and accuracy of analog-to-digital converters (ADC). A popular approach to create wideband ADCs is utilizing multiple channels each operating at a lower speed in the time-interleaved fashion. However, an interleaved ADC comes with its own set of challenges. The parallel architecture is very sensitive to the inter-channel mismatch, timing jitter, clock skew between different ADC channels as well as the nonlinearity within individual channels. Consequently, complex post-calibration is required using digital signal processing (DSP) after the ADC. The traditional DSP calibration consumes a significant amount of power and its design requires knowledge of the source and type of errors which are becoming increasingly difficult to predict in nanometer CMOS processes. In this paper, instead of individually targeting each source of error, we utilize a deep learning algorithm to learn the complete and complex ADC behavior and to compensate for it in realtime. We demonstrate this "Deep ADC" technique on an 8G Sample/s 8-channel time-interleaved ADC with the QAM-OFDM modulated data. Simulation results for different QAM symbol constellations and OFDM subcarriers show dramatic improvements of approximately 5 bits in the dynamic range with a concomitant drastic reduction in symbol error rate. We further discuss the hardware implementation including latency, power consumption, memory requirements, and chip area.

With the crowding of the electromagnetic spectrum and the shrinking cell size in wireless networks, crosstalk between base stations and users is a major problem. Although hand-crafted functional blocks and coding schemes are proven effective to guarantee reliable data transfer, currently deep learning-based approaches have drawn increasing attention in the communication system modeling. In this paper, we propose a Neural Network (NN) based signal processing technique that works with traditional DSP algorithms to overcome the interference problem in realtime. This technique doesn't require any feedback protocol between the receiver and transmitter which makes it very suitable for low-latency and high data-rate applications such as autonomy and augmented reality. While there has been recent work on the use of Reinforcement Learning (RL) in the control layer to manage and control the interference, our approach is novel in the sense that it introduces a neural network for signal processing at baseband data rate and in the physical layer. We demonstrate this "Deep Interference Cancellation" technique using a convolutional LSTM autoencoder. When applied to QAM-OFDM modulated data, the network produces significant improvement in the symbol error rate (SER). We further discuss the hardware implementation including latency, power consumption, memory requirements, and chip area.

THz spectroscopy is an emerging tool for detection of microorganisms and harmful compounds in the food industry, the study of proteins in biomedicine and the development of electron-beam X-ray sources for molecular imaging and lithography. Recording of THz electric field evolution in single-shot is crucially needed in terahertz spectroscopy of irreversible processes in such applications as well as for data communication in the THz portion of the spectrum where there is an abundance of untapped bandwidth. However, achieving sub-picosecond resolution over a long time window has been an open problem for electro-optic sampling -- the standard technique for recording terahertz waveforms. We introduce a new conceptual framework for this open problem that is inspired by time-stretch theory. The novel framework unveils a solution to this 20 year-old problem leading to a dramatic enhancement of the achievable temporal resolution. We validate this new technology in two applications. First, we present single shot recordings of long free-propagating terahertz transients with record time resolution. Second, we present recordings of ultra-short relativistic electron bunches at the European X-ray Free Electron Laser. These results show that electric signals may be now recorded with terahertz bandwidth over arbitrarily long windows, thus enabling the realization of "single-shot terahertz oscilloscopes" and single-shot time-domain spectroscopy systems with an arbitrary time-bandwidth product.

Deep learning has achieved spectacular performance in image and speech recognition and synthesis. It outperforms other machine learning algorithms in problems where large amounts of data are available. In the area of measurement technology, instruments based on the photonic time stretch have established record real-time measurement throughput in spectroscopy, optical coherence tomography, and imaging flow cytometry. These extreme-throughput instruments generate approximately 1 Tbit/s of continuous measurement data and have led to the discovery of rare phenomena in nonlinear and complex systems as well as new types of biomedical instruments. Owing to the abundance of data they generate, time-stretch instruments are a natural fit to deep learning classification. Previously we had shown that high-throughput label-free cell classification with high accuracy can be achieved through a combination of time-stretch microscopy, image processing and feature extraction, followed by deep learning for finding cancer cells in the blood. Such a technology holds promise for early detection of primary cancer or metastasis. Here we describe a new deep learning pipeline, which entirely avoids the slow and computationally costly signal processing and feature extraction steps by a convolutional neural network that directly operates on the measured signals. The improvement in computational efficiency enables low-latency inference and makes this pipeline suitable for cell sorting via deep learning. Our neural network takes less than a few milliseconds to classify the cells, fast enough to provide a decision to a cell sorter for real-time separation of individual target cells. We demonstrate the applicability of our new method in the classification of OT-II white blood cells and SW-480 epithelial cancer cells with more than 95% accuracy in a label-free fashion.

A hallucination-free and computationally efficient algorithm for enhancing the resolution of brain MRI images is demonstrated.

Given prior knowledge of the spectral statistics, the SNR of optical waveforms can be manipulated in a reversible manner. We introduce this concept and discuss its potential application to encryption and context-aware detection of weak signals in noisy environments. The same technology performs single shot sparse sampling for optical data compression.

We introduce a new nonlinear analog optical computing concept that compresses the signal's dynamic range and realizes non-uniform quantization that reshapes and improves the signal-to-noise ratio in the digital domain.

Two-Photon Microscopy has become an invaluable tool for biological and medical research, providing high sensitivity, molecular specificity, inherent three-dimensional sub-cellular resolution and deep tissue penetration. In terms of imaging speeds, however, mechanical scanners still limit the acquisition rates to typically 10-100 frames per second. Here we present a high-speed non-linear microscope achieving kilohertz frame rates by employing pulse-modulated, rapidly wavelength-swept lasers and inertia-free beam steering through angular dispersion. In combination with a high bandwidth, single-photon sensitive detector, we achieve recording of fluorescent lifetimes at unprecedented speeds of 88 million pixels per second. We show diffraction-limited, multi-modal, Two-Photon fluorescence and fluorescence lifetime (FLIM), microscopy and imaging flow cytometry with a digitally reconfigurable laser, imaging system and data acquisition system. These unprecedented speeds should enable high-speed and high-throughput image-assisted cell sorting.

One of the major open problems in computer vision is detection of features in visually impaired images. In this paper, we describe a potential solution using Phase Stretch Transform, a new computational approach for image analysis, edge detection and resolution enhancement that is inspired by the physics of the photonic time stretch technique. We mathematically derive the intrinsic nonlinear transfer function and demonstrate how it leads to (1) superior performance at low contrast levels and (2) a reconfigurable operator for hyper-dimensional classification. We prove that the Phase Stretch Transform equalizes the input image brightness across the range of intensities resulting in a high dynamic range in visually impaired images. We also show further improvement in the dynamic range by combining our method with the conventional techniques. Finally, our results show a method for computation of mathematical derivatives via group delay dispersion operations., Comment: Submitted to PLOS One Journal on November 3, 2016

We show that dispersive propagation of light followed by phase detection has properties that can be exploited for extracting features from the waveforms. This discovery is spearheading development of a new class of physics-inspired algorithms for feature extraction from digital images with unique properties and superior dynamic range compared to conventional algorithms. In certain cases, these algorithms have the potential to be an energy efficient and scalable substitute to synthetically fashioned computational techniques in practice today., Comment: This work has been published in the PHOTOPTICS 2017 - 5th International Conference on Photonics, Optics and Laser Technology, Volume 1, ISBN 978-989-758-223-3, pages 340-345

We are inspired by mechanical gearboxes and demonstrate its analog counterpart in optics. We use nonlinear dispersion modes as "gears" to overcome the mismatch between the ultrahigh speed of optical data and the much slower sampling rate of electronic digitizers and processors. We delineate the mathematical foundations and show its utility in ultrafast optical measurements and digital image compression., Comment: Corrected affiliation

Optical computing accelerators may help alleviate bandwidth and power consumption bottlenecks in electronics. We show an approach to implementing logarithmic-type analog co-processors in silicon photonics and use it to perform the exponentiation operation. The function is realized by exploiting nonlinear-absorption-enhanced Raman amplification saturation in a silicon waveguide.

This paper proposes a new class of hardware accelerators to alleviate bottlenecks in the acquisition, analytics, storage and computation of information carried by wideband streaming signals., Comment: 12 Figures

High throughput real-time instruments are needed to acquire large data sets for detection and classification of rare events. Enabled by the photonic time stretch digitizer, a new class of instruments with record throughputs have led to the discovery of optical rogue waves [1], detection of rare cancer cells [2], and the highest analog-to-digital conversion performance ever achieved [3]. Featuring continuous operation at 100 million frames per second and shutter speed of less than a nanosecond, the time stretch camera is ideally suited for screening of blood and other biological samples. It has enabled detection of breast cancer cells in blood with record, one-in-a-million, sensitivity [2]. Owing to their high real-time throughput, instruments produce a torrent of data - equivalent to several 4K movies per second - that overwhelm data acquisition, storage, and processing operations. This predicament calls for technologies that compress images in optical domain and in real-time. An example of this, based on warped stretch transformation and non-uniform Fourier domain sampling will be reported.

Recently it has been shown that the intensity time-bandwidth product of optical signals can be engineered to match that of the data acquisition instrument. In particular, it is possible to slow down an ultrafast signal, resulting in compressed RF bandwidth - a similar benefit to that offered by the Time-Stretch Dispersive Fourier Transform (TS-DFT) - but with reduced temporal record length leading to time-bandwidth compression. The compression is implemented using a warped group delay dispersion leading to non-uniform time stretching of the signal's intensity envelope. Decoding requires optical phase retrieval and reconstruction of the input temporal profile, for the case where information of interest is resides in the complex field. In this paper, we present results on the general behavior of the reconstruction process and its dependence on the signal-to-noise ratio. We also discuss the role of chirp in the input signal., Comment: Pre-print for Applied Physics Letters submission

Modern computing and data storage systems increasingly rely on parallel architectures where processing and storage load is distributed within a cluster of nodes. The necessity for high-bandwidth data links has made optical communication a critical constituent of modern information systems and silicon the leading platform for creating the necessary optical components. While silicon is arguably the most extensively studied material in history, one of its most important attributes, an analysis of its capacity to carry optical information, has not been reported. The calculation of the information capacity of silicon is complicated by nonlinear losses, phenomena that emerge in optical nanowires as a result of the concentration of optical power in a small geometry. Nonlinear losses are absent in silica glass optical fiber and other common communication channels. While nonlinear loss in silicon is well known, noise and fluctuations that arise from it have never been considered. Here we report sources of fluctuations that arise from two-photon absorption and free-carrier plasma effects and use these results to investigate the theoretical limit of the information capacity of silicon. Our results show that noise and fluctuations due to nonlinear absorption become significant and limit the information capacity well before nonlinear loss itself becomes dominant. We present closed-form analytical expressions that quantify the capacity and provide an intuitive understanding of the underlying interactions. Our results provide the capacity limit and its origin, and suggest solutions for extending it via coding and coherent signaling. The amount of information that can be transmitted by light through silicon is the key element in future information systems. Results presented here are not only applicable to silicon but also to other semiconductor optical channels.

Reduction in the operating voltage of electrooptic modulators is needed in order to remove the bottleneck in the flow of data between electronics and optical interconnects. We report experimental demonstration of a 10 fold reduction in an electrooptic modulator's half-wave voltage up to 50 GHz. This was achieved by employing optical sideband-only amplification.

We introduce the concept of coherent temporal imaging and its combination with the anamorphic stretch transform. The new system can measure both temporal profile of fast waveforms as well as their spectrum in real time and at high-throughput. We show that the combination of coherent detection and warped time-frequency mapping also performs time-bandwidth compression. By reducing the temporal width without sacrificing spectral resolution, it addresses the Big Data problem in real time instruments. The proposed method is the first application of the recently demonstrated Anamorphic Stretch Transform to temporal imaging. Using this method narrow spectral features beyond the spectrometer resolution can be captured. At the same time the output bandwidth and hence the record length is minimized. Coherent detection allows the temporal imaging and dispersive Fourier transform systems to operate in the traditional far field as well as in near field regimes.

A general method for compressing the modulation time-bandwidth product of analog signals is introduced and experimentally demonstrated. As one of its applications, this physics-based signal grooming performs feature-selective stretch, enabling a conventional digitizer to capture fast temporal features that were beyond its bandwidth. At the same time, the total digital data size is reduced. The compression is lossless and is achieved through a same-domain transformation of the signal's complex field, performed in the analog domain prior to digitization. Our method is inspired by operation of Fovea centralis in the human eye and by anamorphic transformation in visual arts. The proposed transform can also be performed in the digital domain as a digital data compression algorithm to alleviate the storage and transmission bottlenecks associated with "big data"., Comment: M. H. Asghari and B. Jalali, "Anamorphic transformation and its application to time-bandwidth compression," submitted on May 28th, 2013 to Applied Optics journal- Accepted. To be presented at IEEE Photonic Conference, September 2013 and in IEEE GlobalSIP, December 2013

Fluorescence imaging is the most widely used method for unveiling the molecular composition of biological specimens. However, the weak optical emission of fluorescent probes and the tradeoff between imaging speed and sensitivity is problematic for acquiring blur-free images of fast phenomena, such as sub-millisecond biochemical dynamics in live cells and tissues, and cells flowing at high speed. We report a solution that achieves real-time pixel readout rates one order of magnitude faster than a modern electron multiplier charge coupled device (EMCCD) - the gold standard in high-speed fluorescence imaging technology. Deemed fluorescence imaging using radiofrequency-multiplexed excitation (FIRE), this approach maps the image into the radiofrequency spectrum using the beating of digitally synthesized optical fields. We demonstrate diffraction-limited confocal fluorescence imaging of stationary cells at a frame rate of 4.4 kHz, as well as fluorescence microscopy in flow at a throughput of approximately 50,000 cells per second., Comment: Main text, 4 figures, supplemental information

As electronic operating frequencies increase toward the terahertz regime, new electrooptic modulators capable of low-voltage high-frequency operation must be developed to provide the necessary optical interconnects. This letter presents a new concept that exploits modulation instability to enhance the intrinsically weak electrooptic effect, \chi^(2). Simulations demonstrate more than 50 times enhancement of electrooptic effect at millimeter wave frequencies leading to a substantial reduction in the required modulation voltage., Comment: 5 page, 4 figures; Submitted for publication January 22, 2013

We show how the ability to slow down, amplify, and capture fast transient events can produce high-throughput real-time instruments ranging from digitizers to imaging flow cytometers for detection of rare diseased cells in blood.