Search

We introduce Chronos, a simple yet effective framework for pretrained probabilistic time series models. Chronos tokenizes time series values using scaling and quantization into a fixed vocabulary and trains existing transformer-based language model architectures on these tokenized time series via the cross-entropy loss. We pretrained Chronos models based on the T5 family (ranging from 20M to 710M parameters) on a large collection of publicly available datasets, complemented by a synthetic dataset that we generated via Gaussian processes to improve generalization. In a comprehensive benchmark consisting of 42 datasets, and comprising both classical local models and deep learning methods, we show that Chronos models: (a) significantly outperform other methods on datasets that were part of the training corpus; and (b) have comparable and occasionally superior zero-shot performance on new datasets, relative to methods that were trained specifically on them. Our results demonstrate that Chronos models can leverage time series data from diverse domains to improve zero-shot accuracy on unseen forecasting tasks, positioning pretrained models as a viable tool to greatly simplify forecasting pipelines., Comment: Code and model checkpoints available at https://github.com/amazon-science/chronos-forecasting

This paper presents non-parametric baseline models for time series forecasting. Unlike classical forecasting models, the proposed approach does not assume any parametric form for the predictive distribution and instead generates predictions by sampling from the empirical distribution according to a tunable strategy. By virtue of this, the model is always able to produce reasonable forecasts (i.e., predictions within the observed data range) without fail unlike classical models that suffer from numerical stability on some data distributions. Moreover, we develop a global version of the proposed method that automatically learns the sampling strategy by exploiting the information across multiple related time series. The empirical evaluation shows that the proposed methods have reasonable and consistent performance across all datasets, proving them to be strong baselines to be considered in one's forecasting toolbox.

Backtracking linesearch is the de facto approach for minimizing continuously differentiable functions with locally Lipschitz gradient. In recent years, it has been shown that in the convex setting it is possible to avoid linesearch altogether, and to allow the stepsize to adapt based on a local smoothness estimate without any backtracks or evaluations of the function value. In this work we propose an adaptive proximal gradient method, adaPG, that uses novel estimates of the local smoothness modulus which leads to less conservative stepsize updates and that can additionally cope with nonsmooth terms. This idea is extended to the primal-dual setting where an adaptive three-term primal-dual algorithm, adaPD, is proposed which can be viewed as an extension of the PDHG method. Moreover, in this setting the "essentially" fully adaptive variant adaPD$^+$ is proposed that avoids evaluating the linear operator norm by invoking a backtracking procedure, that, remarkably, does not require extra gradient evaluations. Numerical simulations demonstrate the effectiveness of the proposed algorithms compared to the state of the art.

Buildings are responsible for a considerable fraction of the energy wasted globally every year, and as a result, excess carbon emissions. While heat is lost directly in colder months and climates, resulting in increased heating loads, in hot climates cooling and ventilation is required. One avenue towards improving the energy efficiency of buildings is to integrate thermoelectric devices and materials within the fabric of the building to exploit the temperature gradient between the inside and outside to do useful work. Cement-based materials are ubiquitous in modern buildings and present an interesting opportunity to be functionalised. We present a systematic investigation of the electronic transport coefficients relevant to the thermoelectric materials of the calcium silicate hydrate (C-S-H) gel analogue, tobermorite, using Density Functional Theory calculations with the Boltzmann transport method. The calculated values of the Seebeck coefficient are within the typical magnitude (200 - 600 $\mu V/K$) indicative of a good thermoelectric material. The tobermorite models are predicted to be intrinsically $p$-type thermoelectric material because of the presence of large concentration of the Si-O tetrahedra sites. The calculated electronic $ZT$ for the tobermorite models have their optimal values of 0.983 at (400 $\mathrm{K}$ and $10^{17}$ $\mathrm{cm^{-3}}$) for tobermorite 9 \r{A}, 0.985 at (400 $\mathrm{K}$ and $10^{17}$ $\mathrm{cm^{-3}}$) for tobermorite 11 \r{A} and 1.20 at (225 $\mathrm{K}$ and $10^{19}$ $\mathrm{cm^{-3}}$) for tobermorite 14 \r{A}, respectively., Comment: 34 pages, 10 figures

This paper introduces a new methodology for detecting anomalies in time series data, with a primary application to monitoring the health of (micro-) services and cloud resources. The main novelty in our approach is that instead of modeling time series consisting of real values or vectors of real values, we model time series of probability distributions over real values (or vectors). This extension to time series of probability distributions allows the technique to be applied to the common scenario where the data is generated by requests coming in to a service, which is then aggregated at a fixed temporal frequency. Our method is amenable to streaming anomaly detection and scales to monitoring for anomalies on millions of time series. We show the superior accuracy of our method on synthetic and public real-world data. On the Yahoo Webscope data set, we outperform the state of the art in 3 out of 4 data sets and we show that we outperform popular open-source anomaly detection tools by up to 17% average improvement for a real-world data set.

Although the performance of popular optimization algorithms such as Douglas-Rachford splitting (DRS) and the ADMM is satisfactory in small and well-scaled problems, ill conditioning and problem size pose a severe obstacle to their reliable employment. Expanding on recent convergence results for DRS and ADMM applied to nonconvex problems, we propose two linesearch algorithms to enhance and robustify these methods by means of quasi-Newton directions. The proposed algorithms are suited for nonconvex problems, require the same black-box oracle of DRS and ADMM, and maintain their (subsequential) convergence properties. Numerical evidence shows that the employment of L-BFGS in the proposed framework greatly improves convergence of DRS and ADMM, making them robust to ill conditioning. Under regularity and nondegeneracy assumptions at the limit point, superlinear convergence is shown when quasi-Newton Broyden directions are adopted.

Deep learning based forecasting methods have become the methods of choice in many applications of time series prediction or forecasting often outperforming other approaches. Consequently, over the last years, these methods are now ubiquitous in large-scale industrial forecasting applications and have consistently ranked among the best entries in forecasting competitions (e.g., M4 and M5). This practical success has further increased the academic interest to understand and improve deep forecasting methods. In this article we provide an introduction and overview of the field: We present important building blocks for deep forecasting in some depth; using these building blocks, we then survey the breadth of the recent deep forecasting literature., Comment: 33 pages, 6 figures

We introduce Gluon Time Series (GluonTS, available at https://gluon-ts.mxnet.io), a library for deep-learning-based time series modeling. GluonTS simplifies the development of and experimentation with time series models for common tasks such as forecasting or anomaly detection. It provides all necessary components and tools that scientists need for quickly building new models, for efficiently running and analyzing experiments and for evaluating model accuracy., Comment: ICML Time Series Workshop 2019

We propose NAMA (Newton-type Alternating Minimization Algorithm) for solving structured nonsmooth convex optimization problems where the sum of two functions is to be minimized, one being strongly convex and the other composed with a linear mapping. The proposed algorithm is a line-search method over a continuous, real-valued, exact penalty function for the corresponding dual problem, which is computed by evaluating the augmented Lagrangian at the primal points obtained by alternating minimizations. As a consequence, NAMA relies on exactly the same computations as the classical alternating minimization algorithm (AMA), also known as the dual proximal gradient method. Under standard assumptions the proposed algorithm possesses strong convergence properties, while under mild additional assumptions the asymptotic convergence is superlinear, provided that the search directions are chosen according to quasi-Newton formulas. Due to its simplicity, the proposed method is well suited for embedded applications and large-scale problems. Experiments show that using limited-memory directions in NAMA greatly improves the convergence speed over AMA and its accelerated variant.

Advances in numerical optimization have supported breakthroughs in several areas of signal processing. This paper focuses on the recent enhanced variants of the proximal gradient numerical optimization algorithm, which combine quasi-Newton methods with forward-adjoint oracles to tackle large-scale problems and reduce the computational burden of many applications. These proximal gradient algorithms are here described in an easy-to-understand way, illustrating how they are able to address a wide variety of problems arising in signal processing. A new high-level modeling language is presented which is used to demonstrate the versatility of the presented algorithms in a series of signal processing application examples such as sparse deconvolution, total variation denoising, audio de-clipping and others., Comment: 30 pages, 14 figures

We present PANOC, a new algorithm for solving optimal control problems arising in nonlinear model predictive control (NMPC). A usual approach to this type of problems is sequential quadratic programming (SQP), which requires the solution of a quadratic program at every iteration and, consequently, inner iterative procedures. As a result, when the problem is ill-conditioned or the prediction horizon is large, each outer iteration becomes computationally very expensive. We propose a line-search algorithm that combines forward-backward iterations (FB) and Newton-type steps over the recently introduced forward-backward envelope (FBE), a continuous, real-valued, exact merit function for the original problem. The curvature information of Newton-type methods enables asymptotic superlinear rates under mild assumptions at the limit point, and the proposed algorithm is based on very simple operations: access to first-order information of the cost and dynamics and low-cost direct linear algebra. No inner iterative procedure nor Hessian evaluation is required, making our approach computationally simpler than SQP methods. The low-memory requirements and simple implementation make our method particularly suited for embedded NMPC applications.

We study the environment-assisted enhancement of the excitation-transport efficiency across a network of interacting quantum particles or sites. Our study reveals a crucial influence of the network configuration - and especially its degree of connectivity - on the amount of environment-supported enhancement. In particular, we find a significant interplay of direct and indirect connections between excitation sending and receiving sites. On the other hand, the non-Markovianity induced by memory-bearing, finite sized environments does not seem to provide a critical resource for the enhanced excitation-transport mechanism.

We introduce a hybrid method for dielectric-metal composites that describes the dynamics of the metallic system classically whilst retaining a quantum description of the dielectric. The time-dependent dipole moment of the classical system is mimicked by the introduction of projected equations of motion (PEOM) and the coupling between the two systems is achieved through an effective dipole-dipole interaction. To benchmark this method, we model a test system (semiconducting quantum dot-metal nanoparticle hybrid). We begin by examining the energy absorption rate, showing agreement between the PEOM method and the analytical rotating wave approximation (RWA) solution. We then investigate population inversion and show that the PEOM method provides an accurate model for the interaction under ultrashort pulse excitation where the traditional RWA breaks down.

We propose ZeroFPR, a nonmonotone linesearch algorithm for minimizing the sum of two nonconvex functions, one of which is smooth and the other possibly nonsmooth. ZeroFPR is the first algorithm that, despite being fit for fully nonconvex problems and requiring only the black-box oracle of forward-backward splitting (FBS) --- namely evaluations of the gradient of the smooth term and of the proximity operator of the nonsmooth one --- achieves superlinear convergence rates under mild assumptions at the limit point when the linesearch directions satisfy a Dennis-Mor\'e condition, and we show that this is the case for quasi-Newton directions. Our approach is based on the forward-backward envelope (FBE), an exact and strictly continuous penalty function for the original cost. Extending previous results we show that, despite being nonsmooth for fully nonconvex problems, the FBE still enjoys favorable first- and second-order properties which are key for the convergence results of ZeroFPR. Our theoretical results are backed up by promising numerical simulations. On large-scale problems, by computing linesearch directions using limited-memory quasi-Newton updates our algorithm greatly outperforms FBS and its accelerated variant (AFBS).

We consider a network of agents, each with its own private cost consisting of the sum of two possibly nonsmooth convex functions, one of which is composed with a linear operator. At every iteration each agent performs local calculations and can only communicate with its neighbors. The goal is to minimize the aggregate of the private cost functions and reach a consensus over a graph. We propose a primal-dual algorithm based on \emph{Asymmetric Forward-Backward-Adjoint} (AFBA), a new operator splitting technique introduced recently by two of the authors. Our algorithm includes the method of Chambolle and Pock as a special case and has linear convergence rate when the cost functions are piecewise linear-quadratic. We show that our distributed algorithm is easy to implement without the need to perform matrix inversions or inner loops. We demonstrate through computational experiments how selecting the parameter of our algorithm can lead to larger step sizes and yield better performance.

The forward-backward splitting method (FBS) for minimizing a nonsmooth composite function can be interpreted as a (variable-metric) gradient method over a continuously differentiable function which we call forward-backward envelope (FBE). This allows to extend algorithms for smooth unconstrained optimization and apply them to nonsmooth (possibly constrained) problems. Since the FBE and its gradient can be computed by simply evaluating forward-backward steps, the resulting methods rely on the very same black-box oracle as FBS. We propose an algorithmic scheme that enjoys the same global convergence properties of FBS when the problem is convex, or when the objective function possesses the Kurdyka-{\L}ojasiewicz property at its critical points. Moreover, when using quasi-Newton directions the proposed method achieves superlinear convergence provided that usual second-order sufficiency conditions on the FBE hold at the limit point of the generated sequence. Such conditions translate into milder requirements on the original function involving generalized second-order differentiability. We show that BFGS fits our framework and that the limited-memory variant L-BFGS is well suited for large-scale problems, greatly outperforming FBS or its accelerated version in practice. The analysis of superlinear convergence is based on an extension of the Dennis and Mor\'e theorem for the proposed algorithmic scheme.

Real-space grids are a powerful alternative for the simulation of electronic systems. One of the main advantages of the approach is the flexibility and simplicity of working directly in real space where the different fields are discretized on a grid, combined with competitive numerical performance and great potential for parallelization. These properties constitute a great advantage at the time of implementing and testing new physical models. Based on our experience with the Octopus code, in this article we discuss how the real-space approach has allowed for the recent development of new ideas for the simulation of electronic systems. Among these applications are approaches to calculate response properties, modeling of photoemission, optimal control of quantum systems, simulation of plasmonic systems, and the exact solution of the Schr\"odinger equation for low-dimensionality systems., Comment: 28 pages, 16 figures

We propose a new approach for analyzing convergence of the Douglas-Rachford splitting method for solving convex composite optimization problems. The approach is based on a continuously differentiable function, the Douglas-Rachford Envelope (DRE), whose stationary points correspond to the solutions of the original (possibly nonsmooth) problem. By proving the equivalence between the Douglas-Rachford splitting method and a scaled gradient method applied to the DRE, results from smooth unconstrained optimization are employed to analyze convergence properties of DRS, to tune the method and to derive an accelerated version of it.

This paper proposes two proximal Newton-CG methods for convex nonsmooth optimization problems in composite form. The algorithms are based on a a reformulation of the original nonsmooth problem as the unconstrained minimization of a continuously differentiable function, namely the forward-backward envelope (FBE). The first algorithm is based on a standard line search strategy, whereas the second one combines the global efficiency estimates of the corresponding first-order methods, while achieving fast asymptotic convergence rates. Furthermore, they are computationally attractive since each Newton iteration requires the approximate solution of a linear system of usually small dimension.

In this article, we report a fully ab initio variational Monte Carlo study of the linear, and periodic chain of Hydrogen atoms, a prototype system providing the simplest example of strong electronic correlation in low dimensions. In particular, we prove that numerical accuracy comparable to that of benchmark density matrix renormalization group calculations can be achieved by using a highly correlated Jastrow-antisymmetrized geminal power variational wave function. Furthermore, by using the so-called "modern theory of polarization" and by studying the spin-spin and dimer-dimer correlations functions, we have characterized in details the crossover between the weakly and strongly correlated regimes of this atomic chain. Our results show that variational Monte Carlo provides an accurate and flexible alternative to highly correlated methods of quantum chemistry which, at variance with these methods, can be also applied to a strongly correlated solid in low dimensions close to a crossover or a phase transition., Comment: 7 pages, 4 figures, submitted to Physical Review B

We present an implementation of Quantum Annealing (QA) via lattice Green's function Monte Carlo (GFMC), focusing on its application to the Ising spin-glass in transverse field. In particular, we study whether or not such method is more effective than the Path-Integral Monte Carlo (PIMC) based QA, as well as classical simulated annealing (CA), previously tested on the same optimization problem. We identify the issue of importance sampling, i.e., the necessity of possessing reasonably good (variational) trial wavefunctions, as the key point of the algorithm. We have considered two possible classes of trial wavefunctions, a mean-field single-site one -- whose optimization is however a very difficult task -- and a Boltzmann-like choice. We performed GFMC-QA runs using such a Boltzmann-like trial wavefunction, finding results for the residual energies that are qualitatively similar to those of CA (but at a much larger computational cost), and definitely worse than PIMC-QA. We conclude that, at present, without a serious effort in constructing reliable importance sampling variational wavefunctions for a quantum glass, GFMC-QA is not a true competitor of PIMC-QA., Comment: 9 pages, 4 figures, REVTex 4

We present results for a variety of Monte Carlo annealing approaches, both classical and quantum, benchmarked against one another for the textbook optimization exercise of a simple one-dimensional double-well. In classical (thermal) annealing, the dependence upon the move chosen in a Metropolis scheme is studied and correlated with the spectrum of the associated Markov transition matrix. In quantum annealing, the Path-Integral Monte Carlo approach is found to yield non-trivial sampling difficulties associated with the tunneling between the two wells. The choice of fictitious quantum kinetic energy is also addressed. We find that a ``relativistic'' kinetic energy form, leading to a higher probability of long real space jumps, can be considerably more effective than the standard one., Comment: 15 pages, 10 figures, REVTex 4