Search

We present an approach for characterizing complex temporal behavior in the sensor measurements of a system in order to support detection of anomalies in that system. We first characterize typical behavior by extending a hidden Markov model-based approach to time series alignment. We then use a trace of that learned behavior to develop a particle filter that enables efficient estimation of the filtering distribution on the state space. This produces filtered residuals that can then be used in an anomaly detection framework. Our motivating example is the daily behavior of a building’s heating, ventilation, and air conditioning (HVAC) system, using sensor measurements that arrive every minute and induce a state space with 15,120 states. We provide an end-to-end demonstration of our approach showing improved performance of anomaly detection after application of alignment and filtering compared to the unaligned data. The proposed model is implemented as a computationally efficient R package alignts (align time series) built with R and Fortran 95 with OpenMP support.

Leveraging the depth and breadth of solutions generated through crowdsourcing can be a powerful accelerator to method development for high consequence problems. While data competitions have...

The Mars rover Curiosity carries an instrument called ChemCam to determine the composition of the soil and rocks via laser-induced breakdown spectroscopy (LIBS). Los Alamos National Laboratory has developed a simulation capability that can predict spectra from ChemCam, but there are major-scale differences between the prediction and observation. This presents a challenge when using Bayesian model calibration to determine the unknown physical parameters that describe the LIBS observations. We present an analysis of LIBS data to support ChemCam based on including a structured discrepancy model in a Bayesian model-calibration scheme. This is both a novel application and an illustration of the importance of setting scientifically informed and constrained discrepancy models within Bayesian model calibration.

The detection, identification, and localization of illicit nuclear materials in urban environments is of utmost importance for national security. Most often, the process of performing these operations consists of a team of trained individuals equipped with radiation detection devices that have built-in algorithms to alert the user to the presence nuclear material and, if possible, to identify the type of nuclear material present. To encourage the development of new detection, radioisotope identification, and source localization algorithms, a dataset consisting of realistic Monte Carlo–simulated radiation detection data from a 2 in. × 4 in. × 16 in. NaI(Tl) scintillation detector moving through a simulated urban environment based on Knoxville, Tennessee, was developed and made public in the form of a Topcoder competition. The methodology used to create this dataset has been verified using experimental data collected at the Fort Indiantown Gap National Guard facility. Realistic signals from special nuclear material and industrial and medical sources are included in the data for developing and testing algorithms in a dynamic real-world background., Measurement(s) gamma ray photon detection events • radiation detection data Technology Type(s) Monte Carlo particle transport model • computational modeling technique Sample Characteristic - Environment city Sample Characteristic - Location State of Tennessee Machine-accessible metadata file describing the reported data: 10.6084/m9.figshare.12654065

The Mars Curiosity rover carries an instrument, ChemCam, designed to measure the composition of surface rocks and soil using laser-induced breakdown spectroscopy (LIBS). The measured spectra from this instrument must be analyzed to identify the component elements in the target sample, as well as their relative proportions. This process, which we call disaggregation, is complicated by so-called matrix effects, which describe nonlinear changes in the relative heights of emission lines as an unknown function of composition due to atomic interactions within the LIBS plasma. In this work we explore the use of the plasma physics code ATOMIC, developed at Los Alamos National Laboratory, for the disaggregation task. ATOMIC has recently been used to model LIBS spectra and can robustly reproduce matrix effects from first principles. The ability of ATOMIC to predict LIBS spectra presents an exciting opportunity to perform disaggregation in a manner not yet tried in the LIBS community, namely via Bayesian model calibration. However, using it directly to solve our inverse problem is computationally intractable due to the large parameter space and the computation time required to produce a single output. Therefore we also explore the use of emulators as a fast solution for this analysis. We discuss a proof of concept Gaussian process emulator for disaggregating two-element compounds of sodium and copper. The training and test datasets were simulated with ATOMIC using a Latin hypercube design. After testing the performance of the emulator, we successfully recover the composition of 25 test spectra with Bayesian model calibration., Comment: 10 pages, 5 figures, special issue

When modeling the reliability of a system or component, it is not uncommon for more than one expert to provide very different prior estimates of the expected reliability as a function of an explanatory variable such as age or temperature. Our goal is to incorporate all information from the experts when choosing a design about which units to test. Bayesian design of experiments has been shown to be very successful for generalized linear models, including logistic regression models. We use this approach to develop methodology for the case where there are several potentially non-overlapping priors under consideration. While multiple priors have been used for analysis in the past, they have never been used in a design context. The Weighted Priors method performs well for a broad range of true underlying model parameter choices and is more robust when compared to other reasonable design choices. We illustrate the method through multiple scenarios and a motivating example. Additional figures for this ar...

Cyber-physical systems - systems that incorporate physical devices with cyber components - are appearing in diverse applications, and due to advances in data acquisition, are accompanied with large amounts of data. The interplay between the cyber and the physical components leaves such systems vulnerable to faults and intrusions, motivating the development of a general model that can efficiently and continuously monitor a cyber-physical system. To be of practical value, the model should be adaptive and equipped with the ability to detect changes in the system. This paper makes three contributions: (1) a new adaptive modeling framework for monitoring an arbitrary cyber-physical system in real-time using a flexible statistical distribution called the normal-gamma; (2) a novel streaming validation procedure, demonstrated on data streams from a cyber-physical system at Los Alamos National Laboratory, to justify the use of the normal-gamma and our new adaptive modeling approach; and (3) a new online change detection algorithm demonstrated on synthetic normal-gamma data streams.

With growing computational capabilities of parallel machines, scientific simulations are being performed at finer spatial and temporal scales, leading to a data explosion. The growing sizes are making it extremely hard to store, manage, disseminate, analyze, and visualize these datasets, especially as neither the memory capacity of parallel machines, memory access speeds, nor disk bandwidths are increasing at the same rate as the computing power. Sampling can be an effective technique to address the above challenges, but it is extremely important to ensure that dataset characteristics are preserved, and the loss of accuracy is within acceptable levels. In this paper, we address the data explosion problems by developing a novel sampling approach, and implementing it in a flexible system that supports server-side sampling and data subsetting. We observe that to allow subsetting over scientific datasets, data repositories are likely to use an indexing technique. Among these techniques, we see that bitmap indexing can not only effectively support subsetting over scientific datasets, but can also help create samples that preserve both value and spatial distributions over scientific datasets. We have developed algorithms for using bitmap indices to sample datasets. We have also shown how only a small amount of additional metadata stored with bitvectors can help assess loss of accuracy with a particular subsampling level. Some of the other properties of this novel approach include: (1) sampling can be flexibly applied to a subset of the original dataset, which may be specified using a value-based and/or a dimension-based subsetting predicate, and (2) no data reorganization is needed, once bitmap indices have been generated. We have extensively evaluated our method with different types of datasets and applications, and demonstrated the effectiveness of our approach.

Problem: Radiation detection systems are deployed at U.S. borders to guard against illicit entry of radioactive material. Unfortunately, nuisance alarms due to naturally occurring radioactive mater...

Detectors to scan for illicit nuclear material began to be installed at various screening locations in 2002. On the sites considered, each vehicle drives slowly by radiation detectors that scan for neutron and gamma radiation, resulting in a time series profile. One performance limitation is that naturally occurring radioactive materials (NORM), such as cat litter, are routinely shipped across borders, leading to nuisance alarms. One strategy for nuisance alarms is to define and recognize "signatures" of certain types of NORM so that many nuisance alarms can be quickly resolved as being innocent. Here, we consider candidate profile features, such as the peak width and the maximum energy ratio, and use pattern recognition methods to illustrate the extent to which several common types of NORM can be distinguished.

Gamma detectors at border crossings are intended to detect illicit nuclear material. These detectors collect counts that are used to determine whether to trigger an alarm. Several candidate alarm rules are evaluated, with attention to background suppression caused by the vehicle. Because the count criterion leads to many nuisance alarms and because background suppression by the vehicle is smaller for ratios of counts, analysis of a ratio criterion is included. Detection probability results that consider the effects of 5 factors are given for 2 signal-injection studies, 1 for counts, and 1 for count ratios.

Prioritization of data is necessary for managing large-scale scientific data, as the scale of the data implies that there are only enough resources available to process a limited subset of the data. For example, data prioritization is used during in situ triage to scale with bandwidth bottlenecks, and used during focus+context visualization to save time during analysis by guiding the user to important information. In this paper, we present ADR visualization, a generalized analysis framework for ranking large-scale data using Analysis-Driven Refinement (ADR), which is inspired by Adaptive Mesh Refinement (AMR). A large-scale data set is partitioned in space, time, and variable, using user-defined importance measurements for prioritization. This process creates a prioritization tree over the data set. Using this tree, selection methods can generate sparse data products for analysis, such as focus+context visualizations or sparse data sets.

As computer simulations continue to grow in size and complexity, they present a particularly challenging class of big data problems. Many application areas are moving toward exascale computing systems, systems that perform $10^{18}$ FLOPS (FLoating-point Operations Per Second) --- a billion billion calculations per second. Simulations at this scale can generate output that exceeds both the storage capacity and the bandwidth available for transfer to storage, making post-processing and analysis challenging. One approach is to embed some analyses in the simulation while the simulation is running --- a strategy often called in situ analysis --- to reduce the need for transfer to storage. Another strategy is to save only a reduced set of time steps rather than the full simulation. Typically the selected time steps are evenly spaced, where the spacing can be defined by the budget for storage and transfer. This paper combines both of these ideas to introduce an online in situ method for identifying a reduced set of time steps of the simulation to save. Our approach significantly reduces the data transfer and storage requirements, and it provides improved fidelity to the simulation to facilitate post-processing and reconstruction. We illustrate the method using a computer simulation that supported NASA's 2009 Lunar Crater Observation and Sensing Satellite mission.

Wavelet analysis is known to be a good option for change detection in many contexts. Detecting changes in solution volumes that are measured with both additive and relative error is an important aspect of safeguards for facilities that process special nuclear material. This paper qualitatively compares wavelet-based change detection to a lag-one differencing option using realistic simulated solution volume data for which the true change points are known. We then show quantitatively that Haar wavelet-based change detection is effective for finding the approximate location of each change point, and that a simple piecewise linear optimization step is effective to refine the initial wavelet-based change point estimate.

With growing computational capabilities of parallel machines, scientific simulations are being performed at finer spatial and temporal scales, leading to a data explosion. The growing sizes are making it extremely hard to store, manage, disseminate, analyze, and visualize these datasets, especially as neither the memory capacity of parallel machines, memory access speeds, nor disk bandwidths are increasing at the same rate as the computing power. Sampling can be an effective technique to address the above challenges, but it is extremely important to ensure that dataset characteristics are preserved, and the loss of accuracy is within acceptable levels. In this paper, we address the data explosion problems by developing a novel sampling approach, and implementing it in a flexible system that supports server-side sampling and data subsetting. We observe that to allow subsetting over scientific datasets, data repositories are likely to use an indexing technique. Among these techniques, we see that bitmap indexing can not only effectively support subsetting over scientific datasets, but can also help create samples that preserve both value and spatial distributions over scientific datasets. We have developed algorithms for using bitmap indices to sample datasets. We have also shown how only a small amount of additional metadata stored with bitvectors can help assess loss of accuracy with a particular subsampling level. Some of the other properties of this novel approach include: 1) sampling can be flexibly applied to a subset of the original dataset, which may be specified using a value-based and/or a dimension-based subsetting predicate, and 2) no data reorganization is needed, once bitmap indices have been generated. We have extensively evaluated our method with different types of datasets and applications, and demonstrated the effectiveness of our approach.

The MATADOR project is focused on developing methods to infer the operational mode of facilities that have the potential to be used in weapons development programs. Our central hypothesis is that by persistent, non-intrusive monitoring of such facilities, differences between various use scenarios can be reliably discovered. The impact of success in this area is that new tools and techniques for monitoring and treaty verification would make it easier to reliably discover and document weapons development activities. This document captures the literature that will serve as a basis to approach this task. The relevant literature is divided into topical areas that relate to the various aspects of expected MATADOR project development. We have found that very little work that is directly applicable for our purposes has been published, which has motivated the development of novel methods under the project. Therefore, the manuscripts referenced in this document were selected based on their potential use as foundational blocks for the methods we anticipate developing, or so that we can understand the limitations of existing methods.

Estimating the noise component of a signal that consists of sinusoids plus broadband noise is a ubiquitous problem. Most methods work in the time-domain, but frequency-domain methods can be computationally more efficient and invariant to the time-domain noise distribution. We compare two prominent frequency-domain approaches, one that computes statistics over periodograms of multiple time segments, and another that computes statistics over frequency segments from a single periodogram. We explore the accuracy-resolution tradeoff for both approaches and provide comparisons of accuracy, sample and segment size dependence, frequency resolution, and computational complexity for each method.

We assess the performance of a sparse classification approach for radiofrequency (RF) transient signals using dictionaries adapted to the data. We explore two approaches: pursuit-type decompositions over analytical, over-complete dictionaries, and dictionaries learned directly from data. Pursuit-type decompositions over analytical, over-complete dictionaries yield sparse representations by design and can work well for target signals in the same function class as the dictionary atoms. Discriminative dictionaries learned directly from data do not rely on analytical constraints or additional knowledge about the signal characteristics, and provide sparse representations that can perform well when used with a statistical classifier. We present classification results for learned dictionaries on simulated test data, and discuss robustness compared to conventional Fourier methods. We draw from techniques of adaptive feature extraction, statistical machine learning, and image processing.

Automatic classification of transitory or pulsed radio frequency (RF) signals is of particular interest in persistent surveillance and remote sensing applications. Such transients are often acquired in noisy, cluttered environments, and may be characterized by complex or unknown analytical models, making feature extraction and classification difficult. We propose a fast, adaptive classification approach based on non-analytical dictionaries learned from data. We compare two dictionary learning methods from the image analysis literature, the K-SVD algorithm and Hebbian learning, and extend them for use with RF data. Both methods allow us to learn discriminative RF dictionaries directly from data without relying on analytical constraints or additional knowledge about the expected signal characteristics. We then use a pursuit search over the learned dictionaries to generate sparse classification features in order to identify time windows that contain a target pulse. In this paper we compare the two dictionary learning methods and discuss how their performance changes as a function of dictionary training parameters. We demonstrate that learned dictionary techniques are suitable for pulsed RF analysis and present results with varying background clutter and noise levels.

Automatic classification of broadband transient radio frequency (RF) signals is of particular interest in persistent surveillance applications. Because such transients are often acquired in noisy, cluttered environments, and are characterized by complex or unknown analytical models, feature extraction and classification can be difficult. We propose a fast, adaptive classification approach based on non-analytical dictionaries learned from data. Conventional representations using fixed (or analytical) orthogonal dictionaries, e.g., Short Time Fourier and Wavelet Transforms, can be suboptimal for classification of transients, as they provide a rigid tiling of the time-frequency space, and are not specifically designed for a particular signal class. They do not usually lead to sparse decompositions, and require separate feature selection algorithms, creating additional computational overhead. Pursuit-type decompositions over analytical, redundant dictionaries yield sparse representations by design, and work well for target signals in the same function class as the dictionary atoms. The pursuit search however has a high computational cost, and the method can perform poorly in the presence of realistic noise and clutter. Our approach builds on the image analysis work of Mairal et al. (2008) to learn a discriminative dictionary for RF transients directly from data without relying on analytical constraints or additional knowledge about the signal characteristics. We then use a pursuit search over this dictionary to generate sparse classification features. We demonstrate that our learned dictionary is robust to unexpected changes in background content and noise levels. The target classification decision is obtained in almost real-time via a parallel, vectorized implementation.

Many problems in electromagnetic signal analysis exhibit dynamics on a wide range of time scales. Further, these dynamics may involve both continuous source generation processes and discrete source mode dynamics. These rich temporal characteristics can present challenges for standard modeling approaches, particularly in the presence of nonstationary noise and clutter sources. Here we demonstrate a hybrid algorithm designed to capture the dynamic behavior at all relevant time scales while remaining robust to clutter and noise at each time scale. We draw from techniques of adaptive feature extraction, statistical machine learning, and discrete process modeling to construct our hybrid algorithm. We describe our approach and present results applying our hybrid algorithm to a simulated dataset based on an example radio beacon identification problem: civilian air traffic control. This application illustrates the multi-scale complexity of the problems we wish to address. We consider a multi-mode air traffic control radar emitter operating against a cluttered background of competing radars and continuous-wave communications signals (radios, TV broadcasts). Our goals are to find a compact representation of the radio frequency measurements, identify which pulses were emitted by the target source, and determine the mode of the source.

Many problems in electromagnetic signal analysis exhibit dynamics on a wide range of time scales against nonstationary clutter and noise. We consider a problem in which the relevant time scales can range from nanoseconds to hours or days (12 or 13 orders of magnitude). We present a hybrid algorithm currently designed to capture the dynamic behavior at scales from nanoseconds to milliseconds (6 orders of magnitude) while remaining robust to clutter and noise. We draw from techniques of adaptive feature extraction, statistical machine learning, and discrete process modeling and present results on a simulated multimode problem. Our goals are to find a representation of the signal that allows us to identify which pulses were produced by a target emitter and to determine the operational mode of the target.

Gamma detectors at border crossings are intended to detect illicit nuclear material. One of their performance challenges is the fact that vehicles suppress the natural background and, thus, potentially reduce probability of detection of threat items. Here we test several methods to adjust the detection to background suppression in the context of signal estimation. We show that, for the small-to-moderate suppression magnitudes, suppression adjustment leads to higher detection probability. However, for signals triggering alarm without an adjustment, adjustment does not improve estimation of the signal location, only moderately improves estimation of the signal magnitude, and does not improve estimation of the signal width.

Measurement of stimulus-induced changes in activity in the brain is critical to the advancement of neuroscience. Scientists use a range of methods, including electrode implantation, surface (scalp) electrode placement, and optical imaging of intrinsic signals, to gather data capturing underlying signals of interest in the brain. These data are usually corrupted by artifacts, complicating interpretation of the signal; in the context of optical imaging, two primary sources of corruption are the heartbeat and respiration cycles. We introduce a new linear state-space framework that uses the Kalman filter to remove these artifacts from optical imaging data. The method relies on a likelihood-based analysis under the specification of a formal statistical model, and allows for corrections to the signal based on auxiliary measurements of quantities closely related to the sources of contamination, such as physiological processes. Furthermore, the likelihood-based modeling framework allows us to perform both goodness-of-fit testing and formal hypothesis testing on parameters of interest. Working with data collected by our collaborators, we demonstrate the method of data collection in an optical imaging study of a cat's brain.

The article introduces this special issue of Technometrics featuring some of the challenging, application-driven work presented at the inaugural Conference on Data Analysis (CoDA) in 2012 in Santa Fe, New Mexico.