Search

Visual line transect (VLT) surveys are central to the monitoring and study of marine mammals. However, for cryptic species such as deep diving cetaceans VLT surveys alone suffer from problems of low sample sizes and availability bias where animals below the surface are not available to be detected. The advent of passive acoustic monitoring (PAM) technology offers important opportunities to observe deep diving cetaceans but statistical challenges remain particularly when trying to integrate VLT and PAM data. Herein, we present a general framework to combine these data streams to estimate abundance when both surveys are conducted simultaneously. Secondarily, our approach can also be used to derive an estimate of availability bias. We outline three methods that vary in complexity and data requirements which are (1) a simple distance sampling (DS) method that treats the two datasets independently (DS-DS Method), (2) a fully integrated approach that applies a capture-mark recapture (CMR) analysis to the PAM data (CMR-DS Method) and (3) a hybrid approach that requires only a subset of the PAM CMR data (Hybrid Method). To evaluate their performance, we use simulations based on known diving and vocalizing behavior of sperm whales (Physeter macrocephalus). As a case study, we applied the Hybrid Method to data from a shipboard survey of sperm whales and compared estimates to a VLT only analysis. Simulation results demonstrated that the CMR-DS Method and Hybrid Method reduced bias by >90% for both abundance and availability bias in comparison to the simpler DS -DS Method. Overall, the CMR-DS Method was the least biased and most precise. For the case study, our application of the Hybrid Method to the sperm whale dataset produced estimates of abundance and availability bias that were comparable to estimates from the VLT only analysis but with considerably higher precision. Integrating multiple sources of data is an important goal with clear benefits. As a step towards that goal we have developed a novel framework. Results from this study are promising although challenges still remain. Future work may focus on applying this method to other deep-diving species and comparing the proposed method to other statistical approaches that aim to combine information from multiple data sources.

In this paper, we propose a coherent framework for multi-machine analysis, using a group clustering model, which can be utilized for predictive maintenance (PdM). The framework benefits from the repetitive structure posed by multiple machines and enables for assessment of health condition, degradation modeling and comparison of machines. It is based on a hierarchical probabilistic model, denoted Gaussian topic model (GTM), where cluster patterns are shared over machines and therefore it allows one to directly obtain proportions of patterns over the machines. This is then used as a basis for cross comparison between machines where identified similarities and differences can lead to important insights about their degradation behaviors. The framework is based on aggregation of data over multiple streams by a predefined set of features extracted over a time window. Moreover, the framework contains a clustering schema which takes uncertainty of cluster assignments into account and where one can specify a desirable degree of reliability of the assignments. By using a multi-machine simulation example, we highlight how the framework can be utilized in order to obtain cluster patterns and inherent variations of such patterns over machines. Furthermore, a comparative study with the commonly used Gaussian mixture model (GMM) demonstrates that GTM is able to identify inherent patterns in the data while the GMM fails. Such result is a consequence of the group level being modeled by the GTM while being absent in the GMM. Hence, the GTM are trained with a view on the data that is not available to the GMM with the consequence that the GMM can miss important, possibly even key cluster patterns. Therefore, we argue that more advanced cluster models, like the GTM, can be key for interpreting and understanding degradation behavior across machines and ultimately for obtaining more efficient and reliable PdM systems.

Bearing faults are the most common failure modes in the rotating system. Vibration data from the rotating system carry important information, that is, characterization and diagnosis; therefore, the vast vibration signals collected from multiple sensors mounted in different sites are transmitted in a certain order for online fault diagnosis. However, due to the influence of transfer paths and noises, the sensitivities to the same fault signal of measured data streams are of significant differences, and signals containing weak sensitivity to the fault are likely to be transmitted preferentially while neglecting transmission order. Meanwhile, high volume vibration data greatly increase the transmission burden. These above-mentioned reasons dramatically reduce online diagnostic efficiency. Thus, fully considering the sensitive differences to the fault for multiple channels, how to transmit measured data streams of multiple sensors for timely online detecting the bearing failure is still a primary challenge. In order to solve this problem, a novel online bearing fault diagnosis method based on the multiple data streams transmission schemes (MDSTS) is proposed in this paper. Multiple sensors are numbered consecutively, and data streams from all channels are transmitted according to the preset order and transport protocol via a certain length at the beginning of diagnosis. Then, a fault sensitivity assessment model (FSAM) is established on maximum mean discrepancy (MMD) for transmitting the most sensitive data stream by calculating the distribution discrepancies between each channel's data streams and the historical datasets in the frequency domain, and then, the fault diagnosis model based on K-nearest neighbor (KNN) trained on historical datasets was used to evaluate the transmission scheme and acquire reliable diagnostic results via predicting performances of multiple and consecutive datablocks until all these exceed an alarm value. The extensive experiment results show that the proposed method can timely and accurately identify the bearing faults and outperforms obviously competitive approaches.

Both multiple-input multiple-output (MIMO) and orthogonal frequency division multiplexing (OFDM) are key technologies for the fifth-generation mobile communication systems. However, interference is one of the critical challenges in MIMO–OFDM systems that degrades the overall performance. In this study, a novel interference alignment (IA) design-based diagonal-block channel charter to mitigate the interference for MIMO–OFDM systems is proposed. The proposed IA scheme can be used in physical layer security, e.g. anti-location. Based on the diagonal-block charter of the channel matrix, the optimal degree of freedom, i.e. the maximum number of multiple data streams concurrently transmitted interference-free can be achieved by IA. Correspondingly, the authors present the closed-form IA design scheme for some special scenarios, and two iterative IA ones for general scenarios in detail. Furthermore, the performance analysis of IA systems with diagonal-block channel error is provided, where the imperfect channel state information case is considered. The simulation results show that the IA design based on the diagonal-block channel can improve the sum-rate performance effectively, by sufficiently exploiting the spatial and frequency resources. [ABSTRACT FROM AUTHOR]

With the rapid development of the World Wide Web technology, complex and diverse data present explosive growth, so frequent itemset mining plays an essential role. In view of the mining frequent itemsets in multiple data streams by limited computing power of a single processor, an improved algorithm of Parallel Mining Collaborative frequent itemsets in multiple data streams (PMCMD-Stream) was proposed. Firstly, the algorithm compresses the potential and frequent itemsets into CP-Tree only by one-scan and applies increment method to inserting or deleting related branch on CP-Tree, we do not need to repeatedly scanning the databases to generate many candidate frequent itemsets and save the running time. Secondly, this parallelized algorithm can be run in the MapReduce programming environment. Finally, the valuable frequent itemsets, namely global collaborative frequent itemsets, were obtained. Because each candidate frequent itemset is independent, and different candidate frequent itemsets can be processed by multiple computing machines concurrently. The experimental results show that PMCMD-Stream algorithm not only can improve the mining efficiency but also have much better scalability than the existing algorithms, so as to discover the collaborative frequent itemsets from large-scale data streams. [ABSTRACT FROM AUTHOR]

Sluice-pump station projects usually consist of many widely distributed hydraulic structures, such as pumping stations, sluices and dams. In order to ensure the safe and reliable operation of the project, it is necessary to observe and measure the settlement, expansion joints and seepage flow of hydraulic structures regularly and accurately. In this paper, an online correlation analysis and clustering method for multichannel real-time monitoring data streams was proposed. It aimed at finding the connections between data streams from multiple interested measuring channels, and automatically discovering potential project security problems and sensor failures. Firstly, the real-time data streams were continuously collected by recording sensor data from multiple measuring channels with the same frequency and aligning them on the time axis. Secondly, 3 statistical features of the data streams were incrementally calculated. By employing the statistical features, the calculation of correlation coefficients of any 2 data streams could only run in 0(1) time. Thirdly, the clustering algorithm of density-based spatial clustering of applications with noise was used in order to automatically find grouped data streams with strong correlations and noised data streams with weak or without correlations. By analyzing the clustering results according to project related characteristics and objective laws, potential project safety risks as well as sensor failures could be identified. Based on an earlier developed safety monitoring system for Taizhou Gaogang sluice-pump station project, the experiments were carried out to analyze and cluster multichannel data streams of uplift pressure, expansion joint and temperature online. It took less than 1 s to process multiple data streams for one time. The clustering results of the water level data streams revealed that the water levels in the uplift pressure tubes installed in 5 sections of the project had strong positive relations owing to the normal action of ground water penetration. Exceptionally, the variation of water level in 1 tube was highly affected by water level change of the Yangtze River, which means there existed an abnormal seepage in that position. The failure of 1 uplift pressure sensor was also found according to the clustering results. Besides, the clustering results of the data streams of expansion joint size and temperature could be explained by thermal expansion and contraction. Especially, the expansion joint sizes of most places in the east-west direction of the horizontal plane had strong negative correlations to the environment temperature while the ones in the other directions were less affected. All the data streams classified as the noises could be directly used to discover the abnormal situations of the corresponding sensors. In conclusion, the proposed method could effectively find the connections between the online data streams from multiple interested measuring channels, and discover potential project safety problems and sensor failures. It showed to be an effective way to supplement the online data analysis methods in the hydraulic area. [ABSTRACT FROM AUTHOR]

Sequential pattern mining is used to find frequent data sequences over time. When sequential patterns are generated, the newly arriving patterns may not be identified as frequent sequential patterns due to the existence of old data and sequences. Progressive sequential pattern mining aims to find the most up-to-date sequential patterns given that obsolete items will be deleted from the sequences. When sequences come with multiple data streams, it is difficult to maintain and update the current sequential patterns. Even worse, when we consider the sequences across multiple streams, previous methods cannot efficiently compute the frequent sequential patterns. In this work, we propose an efficient algorithm PSP-AMS to address this problem. PSPAMS uses a novel data structure PSP-MS-tree to insert new items, update current items, and delete obsolete items. By maintaining a PSP-MS-tree, PSP-AMS efficiently finds the frequent sequential patterns across multiple streams. The experimental results show that PSP-AMS significantly outperforms previous algorithms for mining of progressive sequential patterns across multiple streams on synthetic data as well as real data. [ABSTRACT FROM AUTHOR]

Visual line transect (VLT) surveys are central to the monitoring and study of marine mammals. However, for cryptic species such as deep diving cetaceans VLT surveys alone suffer from problems of low sample sizes and availability bias where animals below the surface are not available to be detected. The advent of passive acoustic monitoring (PAM) technology offers important opportunities to observe deep diving cetaceans but statistical challenges remain particularly when trying to integrate VLT and PAM data. Herein, we present a general framework to combine these data streams to estimate abundance when both surveys are conducted simultaneously. Secondarily, our approach can also be used to derive an estimate of availability bias. We outline three methods that vary in complexity and data requirements which are (1) a simple distance sampling (DS) method that treats the two datasets independently ( DS-DS Method ), (2) a fully integrated approach that applies a capture-mark recapture (CMR) analysis to the PAM data ( CMR-DS Method ) and (3) a hybrid approach that requires only a subset of the PAM CMR data ( Hybrid Method ). To evaluate their performance, we use simulations based on known diving and vocalizing behavior of sperm whales ( Physeter macrocephalus ). As a case study, we applied the Hybrid Method to data from a shipboard survey of sperm whales and compared estimates to a VLT only analysis. Simulation results demonstrated that the CMR-DS Method and Hybrid Method reduced bias by >90% for both abundance and availability bias in comparison to the simpler DS -DS Method . Overall, the CMR-DS Method was the least biased and most precise. For the case study, our application of the Hybrid Method to the sperm whale dataset produced estimates of abundance and availability bias that were comparable to estimates from the VLT only analysis but with considerably higher precision. Integrating multiple sources of data is an important goal with clear benefits. As a step towards that goal we have developed a novel framework. Results from this study are promising although challenges still remain. Future work may focus on applying this method to other deep-diving species and comparing the proposed method to other statistical approaches that aim to combine information from multiple data sources., Competing Interests: Douglas B. Sigourney is employed by Integrated Statistics.

Highlights • A novel synthetic data stream generator (GIDS) was developed. • Use for evaluation of incremental recommender systems and data fusion algorithms. • Resembles real datasets in terms of data properties and recommender performance. • Tunable parameters allow for systematic generation of streams for various problems. Abstract Recommender systems are essential tools in modern e-commerce, streaming services, search engines, social networks and many other areas including the scientific community. However, lack of publicly available data hinders the development and evaluation of recommender algorithms. To address this problem, we propose a Generator of Inter-dependent Data Streams (GIDS), capable of generating multiple temporal and inter-dependent synthetic datasets of relational data. The generator is able to simulate a collection of time-changing data streams, helping to effectively evaluate a variety of recommender systems, data fusion algorithms and incremental algorithms. The evaluation using recommender and data fusion algorithms showed that our generator can successfully mimic real datasets in terms of statistical data properties, and achieved performance of recommender systems. [ABSTRACT FROM AUTHOR]

Identifying the anomalies is a critical task to maintain the uptime of the monitored distributed systems. For this reason, the trace data collected from real time monitors are often provided in form of streams for anomaly detection. Due to the dramatic increase of the scale of modern distributed systems, it is challenging to effectively and efficiently discover the anomalies from a voluminous amount of noisy and high-dimensional data streams. Moreover, the evolving of the system infrastructures brings new anomaly types that cannot be generalized as existing ones, making the existing anomaly detection solutions unavailable. To address these issues, in this paper, we introduce a new type of anomalies called contextual collective anomaly. Then we propose a framework to discover this type of anomaly over a collection of data streams in real time. A primary advantage of this solution is that it can accurately identify the anomalies by taking both the contextual information and the historical information of a data stream into consideration. Also, the proposed framework is designed in a way with a low computational cost, and is able to handle large-scale data streams. To demonstrate the effectiveness and efficiency of our proposed framework, we empirically validate it on a real world cluster. [ABSTRACT FROM AUTHOR]

Temporally and spatially dense data-rich environments provide unprecedented opportunities and challenges for effective process control. In this article, we propose a systematic and scalable adaptive sampling strategy for online high-dimensional process monitoring in the context of limited resources with only partial information available at each acquisition time. The proposed adaptive sampling strategy includes a broad range of applications: (1) when only a limited number of sensors is available; (2) when only a limited number of sensors can be in “ON” state in a fully deployed sensor network; and (3) when only partial data streams can be analyzed at the fusion center due to limited transmission and processing capabilities even though the full data streams have been acquired remotely. A monitoring scheme of using the sum of top-rlocal CUSUM statistics is developed and named as “TRAS” (top-rbased adaptive sampling), which is scalable and robust in detecting a wide range of possible mean shifts in all directions, when each data stream follows a univariate normal distribution. Two properties of this proposed method are also investigated. Case studies are performed on a hot-forming process and a real solar flare process to illustrate and evaluate the performance of the proposed method. [ABSTRACT FROM PUBLISHER]

With the advanced technology of medical devices and sensors, an abundance of medical data streams are available. However, data analysis techniques are very limited, especially for processing massive multiple physiological streams that may only be understood by medical experts. The state-of-the-art techniques only allow multiple medical devices to independently monitor different physiological parameters for the patient's status, thus they signal too many false alarms, creating unnecessary noise, especially in the Intensive Care Unit (ICU). An effective solution which has been recently studied is to integrate information from multiple physiologic parameters to reduce alarms. But it is a challenge to detect abnormalities from high frequently changed physiological streams data, since abnormalities occur gradually due to the complex situation of patients. An analysis of ICU physiological data streams shows that many vital physiological parameters are changed periodically (such as heart rate, arterial pressure, and respiratory impedance) and thus abnormalities are generally abnormal period patterns. In this paper, we develop a Mining Abnormal Period Patterns from Multiple Physiological Streams (MAPPMPS) method to detect and rank abnormalities in medical sensor streams. The efficiency and effectiveness of the MAPPMPS method is demonstrated by a real-world massive database of multiple physiological streams sampled in ICU, comprising 250 patients' streams (each stream involving over 1.3 million data points) with a total size of 28 GB data. [ABSTRACT FROM AUTHOR]

Although issues of data streams have been widely studied and utilized, it is nevertheless challenging to deal with sequential mining of data streams. In this paper, we assume that the transaction of a user is partially coming and that there is no auxiliary for buffering and integrating. We adopt the Path Tree for mining frequent sequential patterns over data streams and integrate the user's sequences efficiently. Algorithms with regards to accuracy (PAlgorithm) and space (PSAlgorithm) are proposed to meet the different aspects of users, although GAlgorithm for mining frequent sequential patterns with a gap limitation is proposed. Many pruning properties are used to further reduce the space usage and improve the accuracy of our algorithms. We also prove that PAlgorithm mine frequent sequential patterns with the approximate support of error guarantee. Through thoughtful experiments, synthetic and real datasets are utilized to verify the feasibility of our algorithms. [ABSTRACT FROM AUTHOR]

Cumulative sum or cusum charts are typically used to detect a change in the distribution of a sequence of observations, e.g., shifts in the mean. Usually, after signalling, the chart is restarted by setting it to some value below the signalling threshold. We propose a non-restarting cusum chart which is able to detect periods during which the stream is out of control. Further, we advocate an upper boundary to prevent the cusum chart rising too high, which helps to detect a change back into control. We present an algorithm to control the false discovery rate when considering cusum charts based on multiple streams of data. We consider two definitions of a false discovery: signalling out-of-control when the observations have been in control since the start and signalling out-of-control when the observations have been in control since the last time the chart was at zero. We prove that the false discovery rate is controlled under both these definitions simultaneously. Simulations reveal the difference in false discovery rate control when using these and other desirable definitions of a false discovery. [ABSTRACT FROM PUBLISHER]

The article presents the novel method to analyze and identify correlated burst patterns through multiple data streams that coevolve overtime. It discusses the use of dynamic probabilistic network to model the dependency structures observed in the data streams. In this regard, it elaborates that the dependencies provides meaningful information concerning the overall system dynamics, which integrated into the burst detection process.

A novel approach to adaptive transmit multi-beamforming is developed for frequency division duplex (FDD) multiple-input multiple-output (MIMO) systems. The method adapts each beam by computing at the receiver the steepest-ascent adaptation direction and feeding the quantized direction vector back to the transmitter; we call it the quantized-feedback optimal adaptive multi-beamforming algorithm. The method is widely flexible in the number of quantization bits and offers an adaptive diversity technique that allows for transmission of multiple data streams on orthogonal beams, thereby substantially increasing the system capacity or providing a new radio link for a new service. The convergence speed is maximized by computing the optimal increment vector which points in the direction of largest directional derivative of the objective function. We show that the algorithm converges globally irrespective of quantization error: the quantized increment vector always provides positive directional derivative at any non-stationary point in the solution space containing no local maxima. Simulation results are presented to illustrate the performance of the method. [ABSTRACT FROM PUBLISHER]

Multimedia systems must deal with multiple data streams. Each data stream usually contains significant volume of redundant noisy data. In many real-time applications, it is essential to focus the computing resources on a relevant subset of data streams at any given time instant and use it to build the model of the environment. We formulate this problem as an experiential sampling problem and propose an approach to utilize computing resources efficiently on the most informative subset of data streams. In this paper, we generalize our experiential sampling framework to multiple data streams and provide an evaluation measure for this technique. We have successfully applied this framework to the problems of traffic monitoring, face detection and monologue detection [ABSTRACT FROM PUBLISHER]