Search

Recurrent Neural Networks excel at predicting and generating complex high-dimensional temporal patterns. Due to their inherent nonlinear dynamics and memory, they can learn unbounded temporal dependencies from data. In a Machine Learning setting, the network's parameters are adapted during a training phase to match the requirements of a given task/problem increasing its computational capabilities. After the training, the network parameters are kept fixed to exploit the learned computations. The static parameters thereby render the network unadaptive to changing conditions, such as external or internal perturbation. In this manuscript, we demonstrate how keeping parts of the network adaptive even after the training enhances its functionality and robustness. Here, we utilize the conceptor framework and conceptualize an adaptive control loop analyzing the network's behavior continuously and adjusting its time-varying internal representation to follow a desired target. We demonstrate how the added adaptivity of the network supports the computational functionality in three distinct tasks: interpolation of temporal patterns, stabilization against partial network degradation, and robustness against input distortion. Our results highlight the potential of adaptive networks in machine learning beyond training, enabling them to not only learn complex patterns but also dynamically adjust to changing environments, ultimately broadening their applicability.

Time delay reservoir computing (TDRC) using semiconductor lasers (SLs) has proven to be a promising photonic analog approach for information processing. One appealing property is that SLs subject to delayed optical feedback and external optical injection, allow tuning the response bandwidth by changing the level of optical injection. Here we use strong optical injection, thereby expanding the SL's modulation response up to tens of GHz. Performing a nonlinear time series prediction task, we demonstrate experimentally that for appropriate operating conditions, our TDRC system can operate with sampling times as small as 11.72 ps, without sacrificing computational performance.

Networks of optically coupled semiconductor lasers are of high interest for fundamental investigations and for enabling numerous technological applications in material processing, lighting and information processing. Still, experimental realizations of large networks of such elements employing a scalable concepts have so far been been lacking. Here, we present a network 22 of the vertical-cavity surface-emitting lasers in a $5 \times 5$ square lattice array via. Crucially, the array allows individual control over each laser's pump current, which we leverage spectrally align the array. Leveraging diffractive coupling through an external cavity, 22 lasers are mutually injection locked, and, furthermore, we demonstrate their simultaneous phase locking to an external injection laser. The VCSEL network is a promising platform for experimental investigations of complex systems and has direct applications as a photonic neural network. The scalability of the concept opens future possibilities for systems comprising many more individual lasers.

Reservoir computing is a machine learning approach that can generate a surrogate model of a dynamical system. It can learn the underlying dynamical system using fewer trainable parameters and hence smaller training data sets than competing approaches. Recently, a simpler formulation, known as next-generation reservoir computing, removes many algorithm metaparameters and identifies a well-performing traditional reservoir computer, thus simplifying training even further. Here, we study a particularly challenging problem of learning a dynamical system that has both disparate time scales and multiple co-existing dynamical states (attractors). We compare the next-generation and traditional reservoir computer using metrics quantifying the geometry of the ground-truth and forecasted attractors. For the studied four-dimensional system, the next-generation reservoir computing approach uses $\sim 1.7 \times$ less training data, requires $10^3 \times$ shorter `warm up' time, has fewer metaparameters, and has an $\sim 100\times$ higher accuracy in predicting the co-existing attractor characteristics in comparison to a traditional reservoir computer. Furthermore, we demonstrate that it predicts the basin of attraction with high accuracy. This work lends further support to the superior learning ability of this new machine learning algorithm for dynamical systems., Comment: 8 pages, 7 figures

We design scalable neural networks adapted to translational symmetries in dynamical systems, capable of inferring untrained high-dimensional dynamics for different system sizes. We train these networks to predict the dynamics of delay-dynamical and spatio-temporal systems for a single size. Then, we drive the networks by their own predictions. We demonstrate that by scaling the size of the trained network, we can predict the complex dynamics for larger or smaller system sizes. Thus, the network learns from a single example and, by exploiting symmetry properties, infers entire bifurcation diagrams.

Reservoir computers are powerful tools for chaotic time series prediction. They can be trained to approximate phase space flows and can thus both predict future values to a high accuracy, as well as reconstruct the general properties of a chaotic attractor without requiring a model. In this work, we show that the ability to learn the dynamics of a complex system can be extended to systems with co-existing attractors, here a 4-dimensional extension of the well-known Lorenz chaotic system. We demonstrate that a reservoir computer can infer entirely unexplored parts of the phase space: a properly trained reservoir computer can predict the existence of attractors that were never approached during training and therefore are labelled as unseen. We provide examples where attractor inference is achieved after training solely on a single noisy trajectory.

Analog photonic computing has been proposed and tested in recent years as an alternative approach for data recovery in fiber transmission systems. Photonic reservoir computing, performing nonlinear transformations of the transmitted signals and exhibiting internal fading memory, has been found advantageous for this kind of processing. In this work, we show that the effectiveness of the internal fading memory depends significantly on the properties of the signal to be processed. Specifically, we demonstrate two experimental photonic post-processing schemes for a 56 GBaud PAM-4 experimental transmission system, with 100 km uncompensated standard single-mode fiber and direct detection. We show that, for transmission systems with significant chromatic dispersion, the contribution of a photonic reservoir's fading memory to the computational performance is limited. In a comparison between the data recovery performances between a reservoir computing and an extreme learning machine fiber-based configuration, we find that both offer equivalent data recovery. The extreme learning machine approach eliminates the necessity of external recurrent connectivity, which simplifies the system and increases the computation speed. Above 31 dB OSNR, the photonics-based equalization exhibits a lower BER than the respective offline DSP-based KK receiver.

Deep neural networks are among the most widely applied machine learning tools showing outstanding performance in a broad range of tasks. We present a method for folding a deep neural network of arbitrary size into a single neuron with multiple time-delayed feedback loops. This single-neuron deep neural network comprises only a single nonlinearity and appropriately adjusted modulations of the feedback signals. The network states emerge in time as a temporal unfolding of the neuron's dynamics. By adjusting the feedback-modulation within the loops, we adapt the network's connection weights. These connection weights are determined via a back-propagation algorithm, where both the delay-induced and local network connections must be taken into account. Our approach can fully represent standard Deep Neural Networks (DNN), encompasses sparse DNNs, and extends the DNN concept toward dynamical systems implementations. The new method, which we call Folded-in-time DNN (Fit-DNN), exhibits promising performance in a set of benchmark tasks.

The efficacy of data decoding in contemporary ultrafast fiber transmission systems is greatly determined by the capabilities of the signal processing tools that are used. The received signal must not exceed a certain level of complexity, beyond which the applied signal processing solutions become insufficient or slow. Moreover, the required signal-to-noise ratio of the received signal can be challenging, especially when adopting modulation formats with multi-level encoding. Lately, photonic reservoir computing (RC) - a hardware machine learning technique with recurrent connectivity - has been proposed as a post-processing tool that deals with deterministic distortions from fiber transmission. Here we show that RC post-processing is remarkably efficient for multilevel encoding and for the use of very high launched optical peak power for fiber transmission up to 14dBm. Higher power levels provide the desired high signal-to-noise ratio (SNR) values at the receiver end, at the expense of a complex nonlinear transformation of the transmission signal. Our demonstration evaluates a direct fiber communication link with 4-level pulse amplitude modulation (PAM-4) encoding and direct detection, without including optical amplification, dispersion compensation, pulse shaping or other digital signal processing (DSP) techniques. By applying RC post-processing on the distorted signal, we numerically estimate fiber transmission distances of 27km at 56Gb/s and of 5.5 km at 112Gb/s data encoding rates, while fulfilling the hard-decision forward error correction (HD-FEC) bit-error-rate (BER) limit for data recovery. In an experimental equivalent demonstration of our photonic reservoir, the achieved distances are 21km and 4.6km respectively.

Photonic Neural Network implementations have been gaining considerable attention as a potentially disruptive future technology. Demonstrating learning in large scale neural networks is essential to establish photonic machine learning substrates as viable information processing systems. Realizing photonic Neural Networks with numerous nonlinear nodes in a fully parallel and efficient learning hardware was lacking so far. We demonstrate a network of up to 2500 diffractively coupled photonic nodes, forming a large scale Recurrent Neural Network. Using a Digital Micro Mirror Device, we realize reinforcement learning. Our scheme is fully parallel, and the passive weights maximize energy efficiency and bandwidth. The computational output efficiently converges and we achieve very good performance.

Machine learning techniques have proven very efficient in assorted classification tasks. Nevertheless, processing time-dependent high-speed signals can turn into an extremely challenging task, especially when these signals have been nonlinearly distorted. Recently, analogue hardware concepts using nonlinear transient responses have been gaining significant interest for fast information processing. Here, we introduce a simplified photonic reservoir computing scheme for data classification of severely distorted optical communication signals after extended fibre transmission. To this end, we convert the direct bit detection process into a pattern recognition problem. Using an experimental implementation of our photonic reservoir computer, we demonstrate an improvement in bit-error-rate by two orders of magnitude, compared to directly classifying the transmitted signal. This improvement corresponds to an extension of the communication range by over 75%. While we do not yet reach full real-time post-processing at telecom rates, we discuss how future designs might close the gap.

In forthcoming years, the Internet of Things (IoT) will connect billions of smart devices generating and uploading a deluge of data to the cloud. If successfully extracted, the knowledge buried in the data can significantly improve the quality of life and foster economic growth. However, a critical bottleneck for realising the efficient IoT is the pressure it puts on the existing communication infrastructures, requiring transfer of enormous data volumes. Aiming at addressing this problem, we propose a novel architecture dubbed Condense, which integrates the IoT-communication infrastructure into data analysis. This is achieved via the generic concept of network function computation: Instead of merely transferring data from the IoT sources to the cloud, the communication infrastructure should actively participate in the data analysis by carefully designed en-route processing. We define the Condense architecture, its basic layers, and the interactions among its constituent modules. Further, from the implementation side, we describe how Condense can be integrated into the 3rd Generation Partnership Project (3GPP) Machine Type Communications (MTC) architecture, as well as the prospects of making it a practically viable technology in a short time frame, relying on Network Function Virtualization (NFV) and Software Defined Networking (SDN). Finally, from the theoretical side, we survey the relevant literature on computing "atomic" functions in both analog and digital domains, as well as on function decomposition over networks, highlighting challenges, insights, and future directions for exploiting these techniques within practical 3GPP MTC architecture., Comment: 17 pages, 7 figures in IEEE Access, Vol. 4, 2016

Nonlinear photonic delay systems present interesting implementation platforms for machine learning models. They can be extremely fast, offer great degrees of parallelism and potentially consume far less power than digital processors. So far they have been successfully employed for signal processing using the Reservoir Computing paradigm. In this paper we show that their range of applicability can be greatly extended if we use gradient descent with backpropagation through time on a model of the system to optimize the input encoding of such systems. We perform physical experiments that demonstrate that the obtained input encodings work well in reality, and we show that optimized systems perform significantly better than the common Reservoir Computing approach. The results presented here demonstrate that common gradient descent techniques from machine learning may well be applicable on physical neuro-inspired analog computers.

We demonstrate reservoir computing with a physical system using a single autonomous Boolean logic element with time-delay feedback. The system generates a chaotic transient with a window of consistency lasting between 30 and 300 ns, which we show is sufficient for reservoir computing. We then characterize the dependence of computational performance on system parameters to find the best operating point of the reservoir. When the best parameters are chosen, the reservoir is able to classify short input patterns with performance that decreases over time. In particular, we show that four distinct input patterns can be classified for 70 ns, even though the inputs are only provided to the reservoir for 7.5 ns., Comment: 5 pages, 5 figures

Nonlinear networks with time-delayed couplings may show strong and weak chaos, depending on the scaling of their Lyapunov exponent with the delay time. We study strong and weak chaos for semiconductor lasers, either with time-delayed self-feedback or for small networks. We examine the dependence on the pump current and consider the question whether strong and weak chaos can be identified from the shape of the intensity trace, the auto-correlations and the external cavity modes. The concept of the sub-Lyapunov exponent $\lambda_0$ is generalized to the case of two time-scale separated delays in the system. We give the first experimental evidence of strong and weak chaos in a network of lasers which supports the sequence 'weak to strong to weak chaos' upon monotonically increasing the coupling strength. Finally, we discuss strong and weak chaos for networks with several distinct sub-Lyapunov exponents and comment on the dependence of the sub-Lyapunov exponent on the number of a laser's inputs in a network.

We show that simultaneous synchronization between two delay-coupled oscillators can be achieved by relaying the dynamics via a third mediating element, which surprisingly lags behind the synchronized outer elements. The zero-lag synchronization thus obtained is robust over a considerable parameter range. We substantiate our claims with experimental and numerical evidence of these synchronization solutions in a chain of three coupled semiconductor lasers with long inter-element coupling delays. The generality of the mechanism is validated in a neuronal model with the same coupling architecture. Thus, our results show that synchronized dynamical states can occur over long distances through relaying, without restriction by the amount of delay., Comment: 10 pages, 4 figures