Search

The explicit regularization and optimality of deep neural networks estimators from independent data have made considerable progress recently. The study of such properties on dependent data is still a challenge. In this paper, we carry out deep learning from strongly mixing observations, and deal with the squared and a broad class of loss functions. We consider sparse-penalized regularization for deep neural network predictor. For a general framework that includes, regression estimation, classification, time series prediction,$\cdots$, oracle inequality for the expected excess risk is established and a bound on the class of H\"older smooth functions is provided. For nonparametric regression from strong mixing data and sub-exponentially error, we provide an oracle inequality for the $L_2$ error and investigate an upper bound of this error on a class of H\"older composition functions. For the specific case of nonparametric autoregression with Gaussian and Laplace errors, a lower bound of the $L_2$ error on this H\"older composition class is established. Up to logarithmic factor, this bound matches its upper bound; so, the deep neural network estimator attains the minimax optimal rate.

Recent developments on deep learning established some theoretical properties of deep neural networks estimators. However, most of the existing works on this topic are restricted to bounded loss functions or (sub)-Gaussian or bounded input. This paper considers robust deep learning from weakly dependent observations, with unbounded loss function and unbounded input/output. It is only assumed that the output variable has a finite $r$ order moment, with $r >1$. Non asymptotic bounds for the expected excess risk of the deep neural network estimator are established under strong mixing, and $\psi$-weak dependence assumptions on the observations. We derive a relationship between these bounds and $r$, and when the data have moments of any order (that is $r=\infty$), the convergence rate is close to some well-known results. When the target predictor belongs to the class of H\"older smooth functions with sufficiently large smoothness index, the rate of the expected excess risk for exponentially strongly mixing data is close to or as same as those for obtained with i.i.d. samples. Application to robust nonparametric regression and robust nonparametric autoregression are considered. The simulation study for models with heavy-tailed errors shows that, robust estimators with absolute loss and Huber loss function outperform the least squares method.

This paper carries out sparse-penalized deep neural networks predictors for learning weakly dependent processes, with a broad class of loss functions. We deal with a general framework that includes, regression estimation, classification, times series prediction, $\cdots$ The $\psi$-weak dependence structure is considered, and for the specific case of bounded observations, $\theta_\infty$-coefficients are also used. In this case of $\theta_\infty$-weakly dependent, a non asymptotic generalization bound within the class of deep neural networks predictors is provided. For learning both $\psi$ and $\theta_\infty$-weakly dependent processes, oracle inequalities for the excess risk of the sparse-penalized deep neural networks estimators are established. When the target function is sufficiently smooth, the convergence rate of these excess risk is close to $\mathcal{O}(n^{-1/3})$. Some simulation results are provided, and application to the forecast of the particulate matter in the Vit\'{o}ria metropolitan area is also considered.

We consider the nonparametric regression and the classification problems for $\psi$-weakly dependent processes. This weak dependence structure is more general than conditions such as, mixing, association, $\ldots$. A penalized estimation method for sparse deep neural networks is performed. In both nonparametric regression and binary classification problems, we establish oracle inequalities for the excess risk of the sparse-penalized deep neural networks estimators. Convergence rates of the excess risk of these estimators are also derived. The simulation results displayed show that, the proposed estimators overall work well than the non penalized estimators.

This paper considers deep neural networks for learning weakly dependent processes in a general framework that includes, for instance, regression estimation, time series prediction, time series classification. The $\psi$-weak dependence structure considered is quite large and covers other conditions such as mixing, association,$\ldots$ Firstly, the approximation of smooth functions by deep neural networks with a broad class of activation functions is considered. We derive the required depth, width and sparsity of a deep neural network to approximate any H\"{o}lder smooth function, defined on any compact set $\mx$. Secondly, we establish a bound of the excess risk for the learning of weakly dependent observations by deep neural networks. When the target function is sufficiently smooth, this bound is close to the usual $\mathcal{O}(n^{-1/2})$.

In this paper, we perform deep neural networks for learning $\psi$-weakly dependent processes. Such weak-dependence property includes a class of weak dependence conditions such as mixing, association,$\cdots$ and the setting considered here covers many commonly used situations such as: regression estimation, time series prediction, time series classification,$\cdots$ The consistency of the empirical risk minimization algorithm in the class of deep neural networks predictors is established. We achieve the generalization bound and obtain a learning rate, which is less than $\mathcal{O}(n^{-1/\alpha})$, for all $\alpha > 2 $. Applications to binary time series classification and prediction in affine causal models with exogenous covariates are carried out. Some simulation results are provided, as well as an application to the US recession data.

We consider statistical learning question for $\psi$-weakly dependent processes, that unifies a large class of weak dependence conditions such as mixing, association,$\cdots$ The consistency of the empirical risk minimization algorithm is established. We derive the generalization bounds and provide the learning rate, which, on some H{\"o}lder class of hypothesis, is close to the usual $O(n^{-1/2})$ obtained in the {\it i.i.d.} case. Application to time series prediction is carried out with an example of causal models with exogenous covariates.

We consider the model selection problem for a large class of time series models, including, multivariate count processes, causal processes with exogenous covariates. A procedure based on a general penalized contrast is proposed. Some asymptotic results for weak and strong consistency are established. The non consistency issue is addressed, and a class of penalty term, that does not ensure consistency is provided. Examples of continuous valued and multivariate count autoregressive time series are considered.

This paper studies the model selection problem in a large class of causal time series models, which includes both the ARMA or AR($\infty$) processes, as well as the GARCH or ARCH($\infty$), APARCH, ARMA-GARCH and many others processes. We first study the asymptotic behavior of the ideal penalty that minimizes the risk induced by a quasi-likelihood estimation among a finite family of models containing the true model. Then, we provide general conditions on the penalty term for obtaining the consistency and efficiency properties. We notably prove that consistent model selection criteria outperform classical AIC criterion in terms of efficiency. Finally, we derive from a Bayesian approach the usual BIC criterion, and by keeping all the second order terms of the Laplace approximation, a data-driven criterion denoted KC'. Monte-Carlo experiments exhibit the obtained asymptotic results and show that KC' criterion does better than the AIC and BIC ones in terms of consistency and efficiency.

We consider an epidemic change-point detection in a large class of causal time series models, including among other processes, AR($\infty$), ARCH($\infty$), TARCH($\infty$), ARMA-GARCH. A test statistic based on the Gaussian quasi-maximum likelihood estimator of the parameter is proposed. It is shown that, under the null hypothesis of no change, the test statistic converges to a distribution obtained from a difference of two Brownian bridge and diverges to infinity under the epidemic alternative. Numerical results for simulation and real data example are provided., Comment: arXiv admin note: text overlap with arXiv:2103.13336

We consider the change-point detection in multivariate continuous and integer valued time series. We propose a Wald-type statistic based on the estimator performed by a general contrast function; which can be constructed from the likelihood, a quasi-likelihood, a least squares method, etc. Sufficient conditions are provided to ensure that the statistic convergences to a well-known distribution under the null hypothesis (of no change) and diverges to infinity under the alternative; which establishes the consistency of the procedure. Some examples are detailed to illustrate the scope of application of the proposed procedure. Simulation experiments are conducted to illustrate the asymptotic results.

In this paper, we consider the structural change in a class of discrete valued time series, which the true conditional distribution of the observations is assumed to be unknown. The conditional mean of the process depends on a parameter $\theta^*$ which may change over time. We provide sufficient conditions for the consistency and the asymptotic normality of the Poisson quasi-maximum likelihood estimator (QMLE) of the model. We consider an epidemic change-point detection and propose a test statistic based on the QMLE of the parameter. Under the null hypothesis of a constant parameter (no change), the test statistic converges to a distribution obtained from a difference of two Brownian bridge. The test statistic diverges to infinity under the epidemic alternative, which establishes that the proposed procedure is consistent in power. The effectiveness of the proposed procedure is illustrated by simulated and real data examples.

In this paper, we study a general class of causal processes with exogenous covariates, including many classical processes such as the ARMA-GARCH, APARCH, ARMAX, GARCH-X and APARCH-X processes. Under some Lipschitz-type conditions, the existence of a $\tau$-weakly dependent strictly stationary and ergodic solution is established. We provide conditions for the strong consistency and derive the asymptotic distribution of the quasi-maximum likelihood estimator (QMLE), both when the true parameter is an interior point of the parameter's space and when it belongs to the boundary. A significance Wald-type test of parameter is developed. This test is quite extensive and includes the test of nullity of the parameter's components, which in particular, allows us to assess the relevance of the exogenous covariates. Relying on the QMLE of the model, we also propose a penalized criterion to address the problem of the model selection for this class. The weak and the strong consistency of the procedure are established. Finally, Monte Carlo simulations are conducted to numerically illustrate the main results.

We consider the strongly consistent question for model selection in a large class of causal time series models, including AR($\infty$), ARCH($\infty$), TARCH($\infty$), ARMA-GARCH and many classical others processes. We propose a penalized criterion based on the quasi likelihood of the model. We provide sufficient conditions that ensure the strong consistency of the proposed procedure. Also, the estimator of the parameter of the selected model obeys the law of iterated logarithm. It appears that, unlike the result of the weak consistency obtained by Bardet {\it et al.} \cite{Bardet2020}, a dependence between the regularization parameter and the model structure is not needed.

We consider together the retrospective and the sequential change-point detection in a general class of integer-valued time series. The conditional mean of the process depends on a parameter $\theta^*$ which may change over time. We propose procedures which are based on the Poisson quasi-maximum likelihood estimator of the parameter, and where the updated estimator is computed without the historical observations in the sequential framework. For both the retrospective and the sequential detection, the test statistics converge to some distributions obtained from the standard Brownian motion under the null hypothesis of no change and diverge to infinity under the alternative; that is, these procedures are consistent. Some results of simulations as well as real data application are provided.

In this article, we study a robust estimation method for a general class of integer-valued time series models. The conditional distribution of the process belongs to a broad class of distribution and unlike classical autoregressive framework, the conditional mean of the process also depends on some multivariate exogenous covariate. We derive a robust inference procedure based on the minimum density power divergence. Under certain regularity conditions, we establish that the proposed estimator is consistent and asymptotically normal. Simulation experiments are conducted to illustrate the empirical performances of the estimator. An application to the number of transactions per minute for the stock Ericsson B is also provided.

We consider an integer-valued time series $Y=(Y_t)_{t\in\Z}$ where the models after a time $k^*$ is Poisson autoregressive with the conditional mean that depends on a parameter $\theta^*\in\Theta\subset\R^d$. The structure of the process before $k^*$ is unknown;? it could be any other integer-valued time series, that is, the process $Y$ could be nonstationary.? It is established that the maximum likelihood estimator of $\theta^*$ computed on the nonstationary observations is consistent and asymptotically normal. Next, we carry out the sequential change-point detection in a large class of Poisson autoregressive models. We propose a monitoring scheme for detecting change in the model. The procedure is based on an updated estimator which is computed without the historical observations. The asymptotic behavior of the detector is studied, in particular, the above result on the inference in a nonstationary setting are applied to prove that the proposed procedure is consistent. A simulation study as well as a real data application are provided.

This paper deals with the problem of model selection for a general class of integer-valued time series. We propose a penalized criterion based on the Poisson quasi-likelihood of the model. Under certain regularity conditions, the consistency of the procedure as well as the consistency and the asymptotic normality of the Poisson quasi-likelihood estimator of the selected model are established. Simulation experiments are conducted for some classical models such as Poisson, binary INGARCH and negative binomial model with nonlinear dynamic. Also, an application to a real dataset is provided.

This paper proposes a piecewise autoregression for general integer-valued time series. The conditional mean of the process depends on a parameter which is piecewise constant over time. We derive an inference procedure based on a penalized contrast that is constructed from the Poisson quasi-maximum likelihood of the model. The consistency of the proposed estimator is established. From practical applications, we derive a data-driven procedure based on the slope heuristic to calibrate the penalty term of the contrast; and the implementation is carried out through the dynamic programming algorithm, which leads to a procedure of $\mathcal{O}(n^2)$ time complexity. Some simulation results are provided, as well as the applications to the US recession data and the number of trades in the stock of Technofirst.

This paper studies the model selection problem in a large class of causal time series models, which includes both the ARMA or AR($\infty$) processes, as well as the GARCH or ARCH($\infty$), APARCH, ARMA-GARCH and many others processes. To tackle this issue, we consider a penalized contrast based on the quasi-likelihood of the model. We provide sufficient conditions for the penalty term to ensure the consistency of the proposed procedure as well as the consistency and the asymptotic normality of the quasi-maximum likelihood estimator of the chosen model. It appears from these conditions that the Bayesian Information Criterion (BIC) does not always guarantee the consistency. We also propose a tool for diagnosing the goodness-of-fit of the chosen model based on the portmanteau Test. Numerical simulations and an illustrative example on the FTSE index are performed to highlight the obtained asymptotic results, including a numerical evidence of the non consistency of the usual BIC penalty for order selection of an AR(p) models with ARCH($\infty$) errors.

We consider the structural change in a class of discrete valued time series that the conditional distribution follows a one-parameter exponential family. We propose a change-point test based on the maximum likelihood estimator of the parameter of the model. Under the null hypothesis (of no change), the test statistics converges to a well known distribution, allowing for the calculation of the critical values of the test. The test statistic diverges to infinity under the alternative, that is, the test asymptotically has power one. Some simulation results and real data applications are reported to show the applicability of the test procedure.

We consider here together the inference questions and the change-point problem in Poisson autoregressions (see Tj{\o}stheim, 2012). The conditional mean (or intensity) of the process is involved as a non-linear function of it past values and the past observations. Under Lipschitz-type conditions, it is shown that the conditional mean can be written as a function of lagged observations. In the latter model, assume that the link function depends on an unknown parameter $\theta_0$. The consistency and the asymptotic normality of the maximum likelihood estimator of the parameter are proved. These results are used to study change-point problem in the parameter $\theta_0$. We propose two tests based on the likelihood of the observations. Under the null hypothesis (i.e. no change), it is proved that both those test statistics converge to an explicit distribution. Consistencies under alternatives are proved for both tests. Simulation results show how those procedure work practically, and an application to real data is also processed., Comment: 35 pages, 5 figures

We propose a new sequential procedure to detect change in the parameters of a process $ X= (X_t)_{t\in \Z}$ belonging to a large class of causal models (such as AR($\infty$), ARCH($\infty$), TARCH($\infty$), ARMA-GARCH processes). The procedure is based on a difference between the historical parameter estimator and the updated parameter estimator, where both these estimators are based on a quasi-likelihood of the model. Unlike classical recursive fluctuation test, the updated estimator is computed without the historical observations. The asymptotic behavior of the test is studied and the consistency in power as well as an upper bound of the detection delay are obtained. Some simulation results are reported with comparisons to some other existing procedures exhibiting the accuracy of our new procedure. The procedure is also applied to the daily closing values of the Nikkei 225, S$&$P 500 and FTSE 100 stock index. We show in this real-data applications how the procedure can be used to solve off-line multiple breaks detection., Comment: arXiv admin note: text overlap with arXiv:1101.5960 by other authors

We consider a process $ X= (X_t)_{t\in \Z}$ belonging to a large class of causal models including AR($\infty$), ARCH($\infty$), TARCH($\infty$),... models. We assume that the model depends on a parameter $\theta_0 \in \R^d$ and consider the problem of testing for change in the parameter. Two statistics $\hat{Q}^{(1)}_n$ and $ \hat{Q}^{(2)}_n$ are constructed using quasi-likelihood estimator (QLME) of the parameter. Under the null hypothesis that there is no change, it is shown that each of these two statistics weakly converges to the supremum of the sum of the squares of independent Brownian bridges. Under the local alternative that there is one change, we show that the test statistic $\hat{Q}_n=\text{max} \big(\hat{Q}^{(1)}_n, \hat{Q}^{(2)}_n \big) $ diverges to infinity. Some simulation results for AR(1), ARCH(1) and GARCH(1,1) models are reported to show the applicability and the performance of our procedure with comparisons to some other approaches., Comment: 18 pages ; two figures ; Submitted to the JTSA

This paper is devoted to the off-line multiple change-point detection in a semiparametric framework. The time series is supposed to belong to a large class of models including AR($\infty$), ARCH($\infty$), TARCH($\infty$),... models where the coefficients change at each instant of breaks. The different unknown parameters (number of changes, change dates and parameters of successive models) are estimated using a penalized contrast built on conditional quasi-likelihood. Under Lipshitzian conditions on the model, the consistency of the estimator is proved when the moment order $r$ of the process satisfies $r\geq 2$. If $r\geq 4$, the same convergence rates for the estimators than in the case of independent random variables are obtained. The particular cases of AR($\infty$), ARCH($\infty$) and TARCH($\infty$) show that our method notably improves the existing results.