Search

This paper applies nonparametric methods to identify jumps in daily futures prices and intraday jumps surrounding inventory announcements of crude oil, heating oil and natural gas contracts traded on the New York Mercantile Exchange. The sample period of our intraday data covers January 1990 to January 2008. We have obtained several interesting empirical results. (1) Large volatility days are often associated with large jump components, and large jump components are often associated with the Energy Information Administration's inventory announcement dates. (2) The volatility jump component is less persistent than the continuous sample path component. (3) Volatility and trading volume are higher on days with a jump at the inventory announcement than on days without a jump at the announcement. Furthermore, the intraday volatility returns to normal faster following inventory announcements with jumps than after announcements without jumps.

Large datasets are often time series data, and such datasets present challenging problems that arise from the passage of time reflected in the datasets. A problem of current interest is clustering and classification of multiple time series. When various time series are fitted to models, the different time series can be grouped into clusters based on the fitted models. If there are different identifiable classes of time series, the fitted models can be used to classify new time series.

Despite advances in sequencing candidate genes and whole genomes, no method has accurately predicted who will or will not benefit from a specific antipsychotic medication among patients with schizophrenia. We propose a computational algorithm that utilizes a person-centered approach that directly identifies individual patients who will respond to a specific antipsychotic medication. The algorithm was applied to the data obtained from the Clinical Antipsychotic Trials of Intervention Effectiveness (CATIE) study. The predictors were either (1) 13 single-nucleotide polymorphisms (SNPs) and 53 baseline variables or (2) 25 SNPs and the same 53 baseline variables, depending on the existing findings and data availability. The outcome variables were either (1) improvement in the Positive and Negative Syndrome Scale (PANSS) (Yes/No) or (2) completion of phase 1/1A (Yes/No). Each of those four predictor-outcome combinations was tried for each of the five antipsychotic medications (Perphenazine, Olanzapine, Quetiapine, Risperidone, and Ziprasidone), leading to 20 prediction experiments. For 18 out of 20 experiments, all three performance measures were greater than 0.50 (sensitivity 0.51-0.79, specificity 0.52-0.79, accuracy 0.52-0.74). Notably, the model provided a promising prediction for Ziprasidone for the case involving completion of phase 1/1A (Yes/No) predicted by 13 SNPs and 53 baseline variables (sensitivity 0.75, specificity 0.74, accuracy 0.74). The proposed algorithm simultaneously used both genetic information and clinical profiles to predict individual patients' response to antipsychotic medications. As the method is not disease-specific but a general algorithm, it can be easily adopted in many other clinical practices for personalized medicine.

Data come in many forms. In the broad view, the term “data” embraces all representations of information or knowledge. There is no single structure that can efficiently contain all of these representations. Some data are in free-form text (for example, the Federalist Papers, which was the subject of a famous statistical analysis), other data are in a hierarchical structure (for example, political units and subunits), and still other data are encodings of methods or algorithms. (This broad view is entirely consistent with the concept of a “stored-program computer”; the program is the data.)

The operations of differentiation and integration of vectors and matrices are logical extensions of the corresponding operations on scalars.

In most applications of linear algebra, problems are solved by transformations of matrices. A given matrix (which represents some transformation of a vector) is itself transformed. The simplest example of this is in solving the linear system Ax = b, where the matrix A represents a transformation of the vector x to the vector b. The matrix A is transformed through a succession of linear operations until x is determined easily by the transformed A and the transformed b.

In this chapter we discuss a wide range of basic topics related to vectors of real numbers. Some of the properties carry over to vectors over other fields, such as complex numbers, but the reader should not assume this. Occasionally, for emphasis, we will refer to “real” vectors or “real” vector spaces, but unless it is stated otherwise, we are assuming the vectors and vector spaces are real. The topics and the properties of vectors and vector spaces that we emphasize are motivated by applications in the data sciences.

In previous chapters, we encountered a number of special matrices, such as symmetric matrices, banded matrices, elementary operator matrices, and so on. In this chapter, we will discuss some of these matrices in more detail and also introduce some other special matrices and data structures that are useful in statistics.

Vectors and matrices are useful in representing multivariate numeric data, and they occur naturally in working with linear equations or when expressing linear relationships among objects. Numerical algorithms for a variety of tasks involve matrix and vector arithmetic. An optimization algorithm to find the minimum of a function, for example, may use a vector of first derivatives and a matrix of second derivatives; and a method to solve a differential equation may use a matrix with a few diagonals for computing differences.

Before we discuss methods for computing eigenvalues, we recall a remark made in Chap. 5 A given nth-degree polynomial p(c) is the characteristic polynomial of some matrix. The companion matrix of equation ( 3.225) is one such matrix.

In this chapter, we build on the notions introduced on page 5, and discuss a wide range of basic topics related to matrices with real elements. Some of the properties carry over to matrices with complex elements, but the reader should not assume this. Occasionally, for emphasis, we will refer to “real” matrices, but unless it is stated otherwise, we are assuming the matrices are real.

One of the most common problems in numerical computing is to solve the linear system $$\displaystyle{Ax = b;}$$ that is, for given A and b, to find x such that the equation holds. The system is said to be consistent if there exists such an x, and in that case a solution x may be written as A−b, where A− is some inverse of A. If A is square and of full rank, we can write the solution as A−1b.

This article describes some recent sensor network research. We have looked in more detail at three data analysis issues. First, we have considered the general issue of what constitutes normal activity and what constitutes anomalous activity. Detecting anomalies requires some measure of ‘normal’ to determine if there is an anomaly. While approaches such as Bayesian networks work well on large and diverse networks, a simpler approach may be useful for smaller networks. We describe an approach for keeping an ongoing record of sensor activation events and turning on an alarm when the frequency of events exceeds a set threshold. The approach is devised for efficiency of computation and storage. We considered groups of days such as seasons, days of the week, and work days to produce several sets of average activation levels. Second, we modeled different time intervals as independent Poisson processes and used this model to compile average data to look for and display anomalies at different sensors and different times. Using a sensor data set, we studied groups of days such as seasons, days of the week, and work days to produce several sets of average activation levels. Third, we consider possible paths that could be caused by an intruder activating several sensors in a spatial and temporal neighborhood. We are developing a simple tool to calculate and display such paths and their probabilities. WIREs Comput Stat 2012, 4:565–570. doi: 10.1002/wics.1233 This article is a U.S. Government work, and as such, is in the public domain in the United States of America.

This paper tackles the issue of building a large-scale virtual organization for individuals and institutions that are associated with the field of computational intelligence (CI) and machine learning (ML). It begins with a few scenarios that will help illustrate the need for a virtual community in CIML.

This paper presents a national-scale map of habitat suitability for tamarisk (Tamarix spp, salt cedar), a high-priority invasive species. We successfully integrate satellite data and tens of thousands of field sampling points through logistic regression modeling to create a habitat suitability map that is 90% accurate. This interagency effort uses field data collected and coordinated through the US Geological Survey and nationwide environmental data layers derived from NASA's MODerate Resolution Imaging Spectroradiometer (MODIS). We demonstrate the use of the map by ranking the 48 continental US states (and the District of Columbia) based on their absolute, as well as proportional, areas of “highly likely” and “moderately likely” habitat for Tamarix. The interagency effort and modeling approach presented here could be used to map other harmful species, in the US and globally.

Statisticians spend a large portion of their working days using the computer. In addition to the standard things that almost everyone does, like E-mail and text processing, and the standard uses teachers make of computers in the classroom, statisticians' use of computers includes data analysis with prepackaged software, development of algorithms and software to implement new statistical methods, Monte Carlo simulation to study the performance of statistical procedures, and mathematical analysis using symbolic processing software. Although the use of the computer for things like E-mail and Web surfing or for classroom demonstrations of the central limit theorem are important, the following comments do not address those activities. I should also make a disclaimer about any specific software package I mention in the following. There are many good software packages in each of the relevant areas of application and of the general types of software; my mention of specific packages is not meant to imply that thos...

Many financial time series are nonstationary and are modeled as ARIMA processes; they are integrated processes (I(n)) which can be made stationary (I(0)) via differencing n times. I(1) processes have a unit root in the autoregressive polynomial. Using OLS with unit root processes often leads to spurious results; a cointegration analysis should be used instead. Unit root tests (URT) decrease spurious cointegration. The Augmented Dickey Fuller (ADF) URT fails to reject a false null hypothesis of a unit root under the presence of structural changes in intercept and/or linear trend. The Zivot and Andrews (ZA) (1992) URT was designed for unknown breaks, but not under the null hypothesis. Lee and Strazicich (2003) argued the ZA URT was biased towards stationarity with breaks and proposed a new URT with breaks in the null. When an ARMA(p,q) process with trend and/or drift that is to be tested for unit roots and has changepoints in trend and/or intercept two approaches that can be taken: One approach is to use a unit root test that is robust to changepoints. In this paper we consider two of these URT's, the Lee-Strazicich URT and the Hybrid Bai-Perron ZA URT(Herranz, 2016.) The other approach we consider is to remove the deterministic components with changepoints using the Bai-Perron breakpoint detection method (1998, 2003), and then use a standard unit root test such as ADF in each segment. This approach does not assume that the entire time series being tested is all I(1) or I(0), as is the case with standard unit root tests. Performances of the tests were compared under various scenarios involving changepoints via simulation studies. Another type of model for breaks, the Self-Exciting-Threshold-Autoregressive (SETAR) model is also discussed.

Monte Carlo simulation is one of the most important tools in all fields of science. Monte Carlo methods use samples of numbers that appear to be random, although the numbers are generated by deterministic computer algorithms. These “pseudorandom” samples can be used in solving a wide range of problems, including both deterministic problems such as evaluating a definite integral and problems in which there is a random component such as studying and comparing the performance of statistical procedures under various assumptions about the underlying distribution. Keywords: Monte Carlo simulation; quadrature; Monte Carlo test; variance reduction