Search

A new method is presented for the numerical identification of a pure random stationary process, a realization of which is given in the form of a time series. The sample covariance function (in the form of serial correlation coefficients) serves as input to an estimation programme which fits it to the autocovariance function of a mixed autoregressive, moving-average (AKMA) model. The fitting is done by maximum likelihood (ML) estimation of the model parameters. An asymptotic theory has been developed, valid for a somewhat more general model. Solution of the ML equation is accomplished by a multivariable Newton—Raphson procedure preceded by a strategic computer search. The estimates, which exist with a probability approaching unity, have the properties of uniqueness, consistency, efficiency, and a joint normal multivariate distribution. The ML method is shown to be asymptotically equivalent to a weighted least squares procedure, accomplished by the minimization of a quadratic form having chi-square distribu...

An autoregressive cum moving average (ARMA) formulation for modelling of surface runoff systems is proposed. It emerges from this study that Nash's model, which has been used successfully in representing British catchments, is a special case of this formulation. The structural relationship between the parameters of the general ARMA and the conceptual formulations has been established. A particular case of the general ARMA formulation has been proposed for the modelling of surface runoff behaviour of the system. The proposed formulation corresponds to a conceptual model of two linear reservoirs in series. This model has been investigated for its stability, nature of response function and range of parameter values.

In recent years there has been increasing interest in autoregressive spectrum estimation. This procedure fits a finite autoregression to the time series data, and calculates the spectrum from the estimated autoregression coefficients and the one step prediction error variance. For multivariate time series, the estimated autoregressive matrices and one step prediction covariance matrix produce estimates of the spectra, coherences, phases, and group delays. The use of Akaike's information criterion (AIC) for identification of the order of the autoregression to be used makes the procedure objective. Experience gained from analyzing large amounts of data from the biological and physical sciences has indicated that AIC works very well for model identification when compared to more subjective procedures such as the examination of partial F -statistics. This experience has also indicated that using both autoregressive spectrum estimation and classical spectrum estimation and superimposing the plots gives a much stronger feeling for the shape of the true spectrum being estimated. The results of some of these analyses are presented.

Mills has estimated a first order, autoregressive distributed lag model of the process of adaptation of urban density gradients. The error term in such equations estimated by ordinary least squares is likely autocorrelated, biasing upward the estimated coefficient of the lagged dependent variable. A specification error may be concealed, with autocorrelation in the error term resulting in overestimates of the length of the adjusted lag. Longer time series samples will be required if the nature of the lagged adjustment processes and the error term are to be simultaneously estimated.

The procedure of maximum‐entropy spectral analysis (MESA), used in the processing of time series data, also applies to wavenumber (bearing) analysis of signals received from a spatially distributed linear array of sensors. The method is precisely the use of autoregressive spectral analysis in the space dimension rather than in time. There are also close links to the predictive deconvolution method used in geophysical work, and to the process of constructing noise‐whitening filters in communication theory, as well as to least‐squares model building. In this note, we review the maximum‐entropy procedure pointing out all these links. The specific algorithm appropriate to a uniformly spaced line array of sensors is given, as well as one possible algorithm for use in the case of nonuniform sensor spacing.

One of the difficulties that arise in the statistical analysis of autoregressive schemes is the very complex nature of the domain of the regression parameters. In the present paper we study an alternative parametrization of autoregressive models of finite order, namely the parametrization by the partial autocorrelations. These are shown to vary freely from −1 to +1 and to be in a one-to-one, continuously differentiable correspondence with the regression parameters. Properties of the asymptotic normal distribution of the maximum likelihood estimates are discussed, and we present a new deduction of Quenouille's result on the asymptotic independence of some of the estimated partial autocorrelations.

CONSIDER THE PROBLEM of maximizing a function f with respect to two variables (or two sets of variables) a1 and a2 within some space S. Suppose it is difficult to maximizefas a function of a1 and a2, but relatively easy to maximizef as a function of a1 given a2 and as a function of a2 given a1. Such a case is frequently encountered in connection with maximum likelihood or quasi-maximum likelihood estimation of certain regression models.2 In this case it is advantageous to adopt a zig-zag iterative procedure which is described below. This procedure was used by Sargan [3] for the purpose of estimating regressions with autoregressive disturbances, but it is amenable to a much more general class of estimation problems. In particular, it can be advantageously used in all cases involving the application of iterative Aitken methods. The plan of the paper is as follows. In Section 2, we present the fundamental lemma and give a proof of convergence in the general case. In Section 3, we demonstrate the applicability of the lemma to the generalized regression model. The last section contains some specific applications that are of interest to econometricians.

Large samples are synthesized through autoregressive, fast fractional noise, and broken line models, based on a practically significant pentad time unit, for four stations in the British Wye catchment. Hydrologic characteristics such as the numbers, durations, and magnitudes of droughts and floods are assessed by means of the crossing properties at critical levels in the time series. If a Gaussian distribution is applied to the stochastic component, crossing properties other than surplus run lengths are unrealistic. Compatibility with historical properties is achieved by the incorporation of approximately twice the coefficient of skewness estimated from the original data through a transformation to a gamma function. Also there is agreement in serial correlograms and coefficients of skewness. In this way neither negative numbers nor outliers are generated through autoregressive models, but run lengths at extreme levels in data from all models need adjustment, and a closer fitting distribution function is desirable.

A least squares predictor is derived to interpolate missing observations of an economic time series in which the recording interval has been shortened. This predictor uses all the available information, but depends on the unknown autocovariance structure of the series. A much simpler predictor, not involving the autocovariances, is shown to be almost optimal whenever the process has the typical economic spectral shape, and simulation experiments indicate that it is superior to the approximate least squares predictor (obtained by estimating the autocovariance structure) and effective in cases exhibiting a seasonal pattern as well as the simple autoregressive case.

SUMMARY The exponential regression model is considered in which the observations arise from a continuous autoregressive process such that the error increment has a normal distribution with zero mean and variance that decreases in proportion to the amount by which the expected value of the observation differs from its asymptote. Various methods of estimating the parameters are considered.

The Box-Jenkins method, a time-based technique for time series analysis, is shown to be successful in modeling chloride and dissolved oxygen data for the St. Clair River near Corunna, Ontario. This is thought to be the first application of the method to water quality data. The technique is demonstrated to be superior in this situation to either a frequency-based approach or a deterministic causative model. The description of the model building process includes the identification, estimation, and diagnostic checking stages. Forecasting and interpretation follow the derivation of the successful models. It is found that an autoregressive type of model best represents the chloride data, and a moving average process the dissolved oxygen following removal of nonstationarity. Similar causative mechanisms appear to influence June and December chloride and June dissolved oxygen.

A number of chart's are presented to facilitate the identification and estimation of the second order moving average and mixed first order autoregressive-moving average models. Two chart's are presented which show the patterns in the correlation and partial correlation functions over the admissible parameter regions for these models to aid in their interpretation and identification. The pure autoregressive form is used to interpret the models. Charts are developed to provide initial estimates of the parameters of t'he models. Methods of obtaining approximate confidence regions by graphs are given and their adequacy discussed. An example is presented to illustrate the use of the charts.

Upper bounds to the capacity of band-limited Gaussian m th-order autoregressive channels with feedback and average energy constraint E are derived. These are the only known hounds on one- and two-way autoregressive channels of order greater than one. They are the tightest known for the first-order case. In this case let \alpha_1 be the regression coefficient, \sigma^2 the innovation variance, N the number of channel iterations per source symbol, and e = E/N ; then the first-order capacity C^1 is bounded by \begin{equation} C^1 \leq \begin{cases} \frac{1}{2} \ln [\frac{e}{\sigma^2}(1+ \mid \alpha_1 \mid ) ^ 2 +1], & \frac{e}{\sigma^2} \leq \frac{1}{1- \alpha_1^2} \\ \frac{1}{2} \ln [\frac{e}{\sigma^2} + \frac{2\mid \alpha_1 \mid}{\sqrt{1-\alpha_1^2}} \sqrt{\frac{e}{\simga^2}} + \frac{1}{1-\alpha_1^2}], & \text{elsewhere}.\\ \end{cases} \end{equation} This is equal to capacity without feedback for very low and very high e/\sigma^2 and is less than twice this one-way capacity everywhere.

Several estimators of the constant mean about which a stationary first-order autoregressive process varies are studied by utilizing Monte Carlo techniques.

Box and Pierce [3] showed that the residual autocorrelations computed from an autoregressive-moving average time series model (single variate) can be successfully utilized for testing the goodness of fit. Here we extend their results to multiple autoregressive time series models. It is shown that the residual autocorrelations can be approximately represented as a singular linear transform of the corresponding white noise autocorrelations and that they possess a multivariate singular normal distribution. Finally a simple, approximate chi-square statistic is proposed to test the fit.

Failure to allow for autocorrelation of the disturbances in a regression model can lead to biased and inconsistent parameter estimates, particularly if the model is autoregressive. While consistent estimation methods are available which allow for autocorrelation, estimation is usually much easier when there is some assurance that autocorrelation is absent. In pursuit of such assurance the present paper develops methods of testing for autocorrelation in two common autoregressive linear regression models. The first model considered is the autoregressive moving average error model (ARMA). It occurs when a first order (two period) moving average of structural disturbances generates the error term of an equation which is to be estimated, and this moving average autocorrelation is taken into account in the process of estimation. A test is developed for the presence of additional Markov autocorrelation of the structural disturbances. The second model considered is the standard autoregressive linear regression model, with a disturbance term which for the purposes of estimation is taken to be serially uncorrelated. Estimation is therefore performed by means of ordinary least squares. A test is developed for the presence of moving average autocorrelation in the disturbance term. In keeping with tradition, both tests are performed by examining the serial correlation of the estimated disturbances. The tests derive from a

A family of linear stochastic difference equation models is proposed for monthly river flows, the variance of the disturbance in the model varying sinusoidally with time. The unknown parameters in the model are estimated by using the given data and an algorithm is given for the choice of the appropriate number and type of autoregressive terms, sinusoidal trend terms, etc., in the model. The method is used to construct a model for the flows in Wabash and other rivers. This model is useful not only for forecasting but also for generating "synthetic flows" since the various statistical characteristics of the output generated by the model such as power spectra, correlograms, monthly means, and variances, etc., are very close to the corresponding characteristics of the observed flows. The model presented in this paper is compared with other models in the literature. The models of the fiver flows constructed by using daily and yearly data and the relative capabilities of the aggregated and nonaggregated models for forecasting are also discussed.

Expressions are obtained for the determinant and inverse of the covariance matrix of a set of n consecutive observations on a mixed autoregressive moving average process. Explicit formulae for the inverse of this matrix are given for the general autoregressive process of order p (n ≧ p), and for the first order mixed autoregressive moving average process.

Several results exposing the interrelations between state-space and frequency-domain descriptions of multivariable linear systems are presented. Three canonical forms for constant parameter autoregressive-moving average (ARMA) models for input-output relations are described and shown to corrrespond to three particular canonical forms for the state variable realization of the model. Invariant parameters for the partial realization problem are characterized. For stochastic processes, it is shown how to construct an ARMA model, driven by white noise, whose output has a specified covariance. A two-step procedure is given, based on minimal realization and Cholesky-factorization algorithms. Though the goal is an ARMA model, it proves useful to introduce an artificial state model and to employ the recently developed Chandrasekhar-type equations for state estimation. The important case of autoregressive processes is studied and it is shown how the Chandrasekhar-type equations can be used to obtain and generalize the well known Levinson-Wiggins-Robinson (LWR) recursion for estimation of stationary autoregressive processes.

Following the convergence proofs for stochastic approximation identification of pure autoregressive (AR) processes with dependent observations, as derived by Saridis and Stein, it is shown that the convergence for mixed autoregressive-moving-average (ARMA) cases can also be proved when none of the AR or the MA parameters or of the covariances are assumed known. Consequently, a generalized stochastic approximations identification procedure for ARMA processes is derived, which is extendable to any linear Kalrman filter models.

The electrical activity of the brain produces a signal which in general is a non-stationary process. Nevertheless stationary Gaussian models are used successfully to characterize the signal in short sections without obvious non-stationarities. However, it has been observed that the characterization of the signal differs more between separate sections than may be explained using a stationary Gaussian model. In this paper a stationary but non-Gaussian process model of EEG is developed and analyzed. It is shown that for a given measurement time estimates of the autocorrelation function and the power spectrum are less accurate for this process than for the corresponding process in a Gaussian model. Upper limits for the increase in the variance of the estimates are given. The process has been simulated on a digital computer and the experimental results verify the theoretical properties.

This paper uses a first-order autoregressive scheme to investigate the effects of initial conditions in a simulation on the estimation of the population mean of a process of interest. The effects are measured by bias and variance. The results show that the elimination of observations near the beginning of the simulation reduces bias, as intended, but increases the variance, sometimes significantly.

In this paper, regression models with error terms generated by a first order autoregressive scheme are analyzed from a Bayesian point of view. Methods are developed for computing posterior distributions of regression coefficients and the parameter of the autoregressive process. The relationship of our approach to sampling theory approaches is briefly discussed.

The problem of estimating the autoregressive parameters of a mixed autoregressive moving-average (ARMA) time series (of known order) using the output data alone is treated. This problem is equivalent to the estimation of the denominator terms of the scalar transfer function of a stationary, linear discrete time system excited by an unobserved unenrrelated sequence input by employing only the observations of the scalar output. The solution of this problem solves the problem of the identification of the dynamics of a white-noise excited continuous-time linear stationary system using sampled data. The latter problem was suggested by Bartlett in 1946. The problem treated here has appeared before in the engineering literature. The earlier treatment yielded biased parameter estimates. An asymptotically unbiased estimator of the autoregressive parameters is obtained as the solution of a modified set of Yule-Walker equations. The asymptotic estimator covariance matrix behaves like a least-squares parameter estimate of an observation set with unknown error covariances. The estimators are also shown to be unbiased in the presence of additive independent observation noise of arbitrary finite correlation time. An example illustrates the performance of the estimating procedures.

This paper deals with the finite part of multiple autoregressive series. The conditions of stationarity are derived and the inverse of the covariance matrix is evaluated (without assumption of stationarity).

The problem considered requires estimation of parameters characterizing a serially correlated hydrologic time series {yt}, where data are also available from a longer time series {xt}, itself serially correlated and cross correlated with {yt}. On the assumption that both series are lag one autoregressions (when {yt} is characterized by three parameters μy, β, and σϵ2), large-sample variances for the autoregressive parameter β are derived from the short series {yt} alone or from both series {xt} and {yt}, and the relative information is investigated numerically. It is concluded that, when the two serial correlations are approximately equal and the length of {xt} is twice that of {yt}, the gain in precision of the estimate β is about 4% when the cross correlation is 0.2, about 15% when the cross correlation is 0.4, about 31% when the cross correlation is 0.6, and about 49% when the cross correlation is 0.8. The equations giving maximum likelihood (ML) estimates are examined, and a relatively simple numerical technique is developed for one particular case of some practical importance. Small-sample properties of both ML estimates and an estimate derived from work by Matalas are investigated, and tentative conclusions are that (1) the modified Matalas estimates have smaller mean square error than ML estimates, and (2) ML estimates of μy and β derived from both series have smaller mean square error than ML estimates of μy and β derived from the single series only, the converse being true for σϵ2.

SUMMARY This paper describes a Bayesian procedure for the simultaneous estimation of the proba- bilities in a histogram. A two-stage prior distribution is constructed which assumes that probabilities corresponding to adjacent intervals are likely to be closely related. The method employs multivariate logit transformations, and a covariance structure similar to that assumed in the first-order autoregressive process. Posterior estimates are obtained which combine information between the intervals and have the practical effect of smoothing the histogram. A weakness of the Bayesian approach has been its inability to cope with independent observations whose common distribution is not restricted to any particular family. We seek to remedy this deficiency by providing a technique for the analysis of n observations, which are assumed independent and identically distributed with unknown density q(y) which is concentrated on a finite interval I of the real line. We will assume that q(y) is thought a priori to possess a continuous first derivative for all y eI, or to possess some similar property of smoothness. We are posed with the problem of how to obtain estimates for q(y) and its moments which take account of this prior informa- tion. The problem will be treated by using a histogram to approximate q(y), and by esti- mating the probabilities in the histogram under the assumption that they are related in a certain manner. A disadvantage of our method is that the histogram estimate for q(y) will be discontinuous at several points in 1, and will not usually satisfy the smoothness property assumed a priori for the theoretical density. However, we hope that this will to some extent be compensated for by the advantages of the estimation procedure proposed for the probabilities in the histogram. Good & Gaskins (1971) and Boneva, Kendall & Stefanov (1971) provided sampling theory methods for the estimation of a density. An advantage of a Bayesian approach is that it takes proper account of the prior information, since the latter may be incorrectly emphasized when basing the analysis on intuitive ideas. Our method will be fairly flexible in allowing information about the degree of smoothness, and the shape, of the density to be incorporated into the prior model.

Exact tests for serial independence in vector Markoff processes have been obtained by considering the system of regressions of the vectors observed at time 2t on the vectors observed at times (2t − 1) and (2t + 1). The tests then reduce to those obtained from canonical correlation procedures in multivariate analysis. Two particular cases are(1) The test for serial independence of a vector of residuals from a system of regressions in which the regressors are all independent of the residuals. At the same time a test of the hypothesis that the regressor and regrediend vectors are independent is obtained.(2) The test for serial correlation or partial serial correlation in a multiple Markoff process (autoregressive process).An investigation of the efficiency of the test so obtained, of the hypothesis that a process is a simple Markoff process (against the alternative that the process is a second-order auto-regression) suggests that the efficiency of all of these tests will be low.

A predictive model, employing a first order autoregressive structure, which enables estimation of the probability distributions of streamflow volumes in future seasons is developed. The model is applicable to streams that have intraseasonal runoff volumes describable by lognormal probability distributions. A season long observation provides the initial value which is propagated through future seasons by the first order autoregressive streamflow volume structure. For high interseasonal (lag one) correlations application of the model yields valuable forecasts for as many as 10 time periods in the future, while for low correlations such forecasting is of no additional value after four time periods. The method has the greatest utility for streams having high variability and distinct seasonality.

Two Bayesian autoregressive time series models for partially observable dynamic processes are presented. In the first model, a general inference procedure is developed for the situation in which k previous values of the time series plus a change error determine the next value. This general model is specialized to an example in which the observational and change errors follow a normal probability law; the results for k = 1 are given and discussed. The second general model adds the facility for simultaneously inferring an unknown and unchanging parameter of the time series. This model is specialized to the same normal example presented earlier, with the precision of the change error as the unknown process parameter.

This study concerns the relative efficiency of the least-squares estimator of the mean of a covariance stationary sequence that has a constant mean and an autoregressive representation of order p. Our results augment those in [2]. The results in Section 1 include for any positive integer p the variance of the LSE, the BLUE in terms of autoregressive coefficients, the variance of the BLUE and the range of applications of Ylvisaker's theorem [7] regarding a lower bound on relative efficiency. For second-order schemes it also includes bounds on parameter values for relative efficiency >0.80. Graphical results for the first-order scheme in Section 2 add to the detailed study made in [2] and imply that the relative desirability of the LSE should be based on variance as well as relative efficiency considerations.

A first-order linear model for responses measured on a continuous scale is considered. Expressions are derived for stimulus dependencies, autocorrelations, and variances, thus giving a unified treatment of previous work. The stimulus dependencies are shown to remain essentially unchanged for certain cases of nonconstant parameters. The analysis of continuous contingent schedules is illustrated. A higher-order linear model is also discussed briefly. It is shown that the stimulus dependencies can provide parameter-free tests of the whole class of models under consideration. Relations of these models to autoregressive models, and to discreteresponse, linear operator models are pointed out.

The paper compares the power of two tests for serial correlation in regression models with lagged dependent variables, recently suggested by Durbin, with that of the likelihood ratio test by means of two sets of Monte-Carlo experiments-one in which the exogenous series is taken to be the quarterly GNP series for the USA and the other in which the exogenous series is generated by a known autoregression.

The problem of identification of a process under a feedback control system of Box and Jenkins type is discussed and various methods considered for updating the control system. Deliberate system perturbation is unnecessary since excitation is provided by the inherent disturbances. A predictor updating scheme is shown, if convergent, to result in an optimum system. However its performance is crucially dependent on the availability of a good process model. A more promising technique relies on a separation effect applicable to processes with time delays in excess of three sample intervals, based on the assumption that the disturbance is adequately described by an autoregressive model of three or fewer coefficients. The technique requires the fitting of an autoregressive model by the linear least squares method and has proved practically reliable. The more general problem is discussed using Astrom's formulation and a general technique examined. Implementation of such systems requires engineering judgement, typified by the discussion on steady-state errors.

sider the problem of estimating the set of parameters (ca,, ..., op, /,1 ... ,/&), given consecutive observations (xl, x2, ..., xn) from the series, and show that for large n, the method of estimation for a moving-average model (obtained by putting coi = 0, 1 < i < p) which is described in a previous paper (Walker, 1961 b), referred to subsequently as I, can readily be adapted to the above model. As in I we shall assume that the representation of the moving-average residual process is 'regular', i.e. that the moduli of the roots of the equation

Past work on prediction formulas for time series, e.g., that of Wiener and Yaglom, has used powerful and general spectral techniques that result in integral formulas difficult to evaluate in practical cases. In this paper the more direct, less powerful approach developed by Muth produces useful and specific results for a broad class of cases that includes virtually every case of a higher order Markov process observed with error. The results provide plausible theoretical support for the expectations model widely and successfully applied by economists, although the distributions of the estimators of the prediction parameters (when the latter are unknown) are still unknown. The two-hundred-year series of sunspot data provides an illustrative application of the theoretical results, which bears comparison with the previous analyses of these data by Yule and others.

LAND PRICES as an average for the United States have increased over 100 percent since 1950. Meanwhile prices received for commodities sold by farmers have declined 8 percent and net farm income has remained nearly stable. This paper contains an econometric model which can be used to trace the sources of land price increments over a period of time. By imputing the price increments (for example since 1950) to reasonably stable factors such as scale economies or to unstable factors such as speculation, the model can help to answer the question "Are land prices too high?" The land model is estimated by ordinary, recursive, and autoregressive least squares. The equations are estimated with annual national time series for 1923-1963, but the results are directed primarily toward analysis of the land price rise since 1950.

The full text of the paper describes in detail: 1) the identification of the time series model from autocorrelation and partial correlation of the data; 2) the estimation of the , 6, ramp and random walk parameters using maximum likelihood and nonlinear least squares; and 3) the means by which one would deduce model adequacy through autocorrelation of the white noise residuals and confidence limit theory. As an example of the theory, consider Fig. 1, which shows a sample of normalized long term gyro drift rate. Since the process is nonstationary, the data is differenced as is shown in Fig. 2. The analysis indicated that the math model for this gyro drift rate sample is

A method for stochastic population projection for small areas is developed for use by consulting engineers as a basis for specifying the design capacity of pollution control facilities. Based on the components-of-change model for population growth and a formulation of demographic rates as first-order autoregressive processes, the proposed projection model specifies future population in terms of a probability distribution. Monte Carlo simulation rather than analytical solution is used to derive the probabilities. An analysis of the first results suggests a significant improvement in predictive accuracy over the traditional subjective or deterministic projection models still in widespread use by the environmental engineering profession.