Search

Complex event monitoring is an important problem in data streams that has drawn much attention. Most previous work assumes that the user knows and provides a complex event pattern for the system to continuously monitor. However, we observe that in many real applications, such as healthcare, security, and businesses, there are heterogeneous substreams and a diverse set of attributes. Often there is no simple uniform pattern prior to a critical event; nor is there clean simple language to describe the pattern leading to the critical event. People often only know it after the fact -- e.g., when something undesirable happens. We propose a novel approach based on relational machine learning and representation learning. We propose and learn probabilistic state machine patterns, which are used to monitor and predict the imminence of critical events. Our experiments demonstrate the efficiency and effectiveness of our approach, as well as its clear superiority over the closest previous approaches such as IL-Miner and LSTM based early prediction.

Traffic analysis attacks are common in monitoring wireless sensor networks (WSNs). Therefore, in this study, we explore the features of various adversaries who present threats of traffic analysis attacks against source node location privacy (SLP) routing protocols. Subsequently, we present a series of experiments to analyze various adversary attacks. Analysis results show that an enhanced hotspot-locating adversary (EH-LA) is a big threat against SLP protocols. When compared to patient adversary (PA) and hotspot-locating adversary (H-LA), the EH-LA poses significantly stronger attacks. The results also show that the strength of the PA, H-LA, and EH-LA attacks can improve when factors such as the network operation duration, adversary hearing range, and source node packet rate are varied. On the other hand, factors such as increased source-sink distance, node density, and network size have shown to impede the adversaries attack success.

Event monitoring and detection in real-time systems is crucial. Protecting users' data while reporting an event in almost real-time will increase the level of this challenge. In this work, we adopt the strong notion of differential privacy to private stream counting for event detection with the aim of minimizing false positive and false negative rates as our utility metrics.

Individuals are continually observed by an ever-increasing number of sensors that make up the Internet of Things. The resulting streams of data, which are analyzed in real time, can reveal sensitive personal information about individuals. Hence, there is an urgent need for stream processing solutions that can analyze these data in real time with provable guarantees of privacy and low error. We present PeGaSus, a new algorithm for differentially private stream processing. Unlike prior work that has focused on answering individual queries over streams, our algorithm is the first that can simultaneously support a variety of stream processing tasks -- counts, sliding windows, event monitoring -- over multiple resolutions of the stream. PeGaSus uses a Perturber to release noisy counts, a data-adaptive Perturber to identify stable uniform regions in the stream, and a query specific Smoother, which combines the outputs of the Perturber and Grouper to answer queries with low error. In a comprehensive study using a WiFi access point dataset, we empirically show that PeGaSus can answer continuous queries with lower error than the previous state-of-the-art algorithms, even those specialized to particular query types.

As data centers grow larger and strive to provide tight performance and availability SLAs, their monitoring infrastructure must move from passive systems that provide aggregated inputs to human operators, to active systems that enable programmed control. In this paper, we propose Trumpet, an event monitoring system that leverages CPU resources and end-host programmability, to monitor every packet and report events at millisecond timescales. Trumpet users can express many *network-wide events*, and the system efficiently detects these events using *triggers* at end-hosts. Using careful design, Trumpet can evaluate triggers by inspecting every packet at full line rate even on future generations of NICs, scale to thousands of triggers per end-host while bounding packet processing delay to a few microseconds, and report events to a controller within 10 milliseconds, even in the presence of attacks. We demonstrate these properties using an implementation of Trumpet, and also show that it allows operators to describe new network events such as detecting correlated bursts and loss, identifying the root cause of transient congestion, and detecting short-term anomalies at the scale of a data center tenant.

This work presents a distributed implementation for processing Maximizing Range Sum (MaxRS) query in Wireless Sensor Networks (WSN). MaxRS query is useful in many spatially-distributed event monitoring and target tracking applications. Given the location and current readings of the nodes, and a rectangle R, MaxRS finds a location of R that maximizes the sum of the readings of all the nodes covered by R. Our system performs MaxRS query in a user-specified time-interval γ and using the result obtained, attempts to maintain a certain degree of energy conservation in the WSN, based on a user-defined threshold δ. Since centralized processing of the raw readings and subsequently determining the MaxRS may incur significant communication overheads, we developed a distributed algorithm to compute MaxRS. We implemented our system in a heterogeneous WSN consisting of TelosB and SunSPOT motes, and illustrate the end-user tools: GUI for specifying required parameters, and real-time visualization of MaxRS solutions and estimated network energy consumption.

As high-performance computing (HPC) systems become larger and more complex, fault tolerance becomes a greater concern. At the same time, the data volume collected to help in understanding and mitigating hardware and software faults and failures also becomes prohibitively large. We argue that the HPC community must adopt more systematic approaches to system event logging as opposed to the current, ad hoc, strategies based on practitioner intuition and experience. Specifically, we show that event correlation and prediction can increase our understanding of fault behavior and can become critical components of effective fault tolerance strategies. While event correlation and prediction have been used in HPC contexts, we offer new insights about their potential capabilities. Using event logs from the computer failure data repository (cfdr) (1) we use cross and partial correlations to observe conditional correlations in HPC event data; (2) we use information theory to understand the fundamental predictive power of HPC failure data; (3) we study neural networks for failure prediction; and (4) finally, we use principal component analysis to understand to what extent dimensionality reduction can apply to HPC event data. This work results in the following insights that can inform HPC event monitoring: ad hoc correlations or ones based on direct correlations can be deficient or even misleading; highly accurate failure prediction may only require small windows of failure event history; and principal component analysis can significantly reduce HPC event data without loss of relevant information.

Effective Big Data analytics pose several difficult challenges for modern data management architectures. One key such challenge arises from the naturally streaming nature of big data, which mandates efficient algorithms for querying and analyzing massive, continuous data streams (that is, data that is seen only once and in a fixed order) with limited memory and CPU-time resources. Such streams arise naturally in emerging large-scale event monitoring applications; for instance, network-operations monitoring in large ISPs, where usage information from numerous sites needs to be continuously collected and analyzed for interesting trends. In addition to memory- and time-efficiency concerns, the inherently distributed nature of such applications also raises important communication-efficiency issues, making it critical to carefully optimize the use of the underlying network infrastructure. In this talk, we introduce the distributed data streaming model, and discuss recent work on tracking complex queries over massive distributed streams, as well as new research directions in this space.

During the past years the interest in the exploitation of mobility information has increased significantly. A growing number of companies and research institutions are interested in the analysis of mobility data with demand of a high level of spatial detail. Means of tracking persons in our environment can nowadays be fulfilled by utilizing several technologies, for example the Bluetooth technology, offering means to obtain movement data. This paper gives an overview of four case studies in the field of Bluetooth tracking which were conducted in order to provide helpful insights on movement aspects for decision makers in their specific microcosm. Aim is to analyse spatio-temporal validity of Bluetooth tracking, and in doing so, to describe the potential of Bluetooth in pedestrian mobility mining.

Despite the multitude of protocols that have been designed for event monitoring in sensor networks, very little work has been devoted to assessing the quality of these protocols in terms of their ability to provide timely and accurate information about events. The problem is challenging because of the combined effects of the stochastic nature of event arrival and duration and the stochastic nature of a sensor's sleep-awake schedule. These various stochastic processes need to be modelled jointly in order to be able to reason, with any degree of confidence, about the quality of the event detection. The main contribution of this paper is a framework for assessing our confidence about the occurrence of events in sensor networks. Our framework models the interaction between the stochastic nature of an event arrival, the event duration, and the sensor sleep/awake schedule. By employing our framework, it is possible to configure individual sensor duty cycles to meet the requirements of mission-oriented applications in terms of timely and accurate information about events of interest.

The primary objective of various data stream applications is to monitor the on-going variations of data elements newly generated in data streams, so that appropriate services can be delivered to users timely. Basically, such monitoring activities can be divided into three categories: instant event monitoring, frequent behavior monitoring and analytic OLAP measures monitoring. Several prototype systems for data streams have been introduced but they are designated to support only the operations of one or two categories. However, many real-world data stream applications often require mixed operations of the three categories. This demonstration introduces an Integrated Stream Execution Environment (i-SEE) that can support the mixed execution of the operations of the three categories seamlessly, so that an i-SEE system can provide the multifaceted analytic results of on-line data streams continuously for various emerging applications.

MapReduce is a popular framework for data-intensive distributed computing of batch jobs. To simplify fault tolerance, the output of each MapReduce task and job is materialized to disk before it is consumed. In this demonstration, we describe a modified MapReduce architecture that allows data to be pipelined between operators. This extends the MapReduce programming model beyond batch processing, and can reduce completion times and improve system utilization for batch jobs as well. We demonstrate a modified version of the Hadoop MapReduce framework that supports online aggregation, which allows users to see "early returns" from a job as it is being computed. Our Hadoop Online Prototype (HOP) also supports continuous queries, which enable MapReduce programs to be written for applications such as event monitoring and stream processing. HOP retains the fault tolerance properties of Hadoop, and can run unmodified user-defined MapReduce programs.

In an event-monitoring wireless sensor network (WSN), the sensors must be waked up fast to engage in active and high-rate sensing when events of interest occur. However, to prolong the lifetime of the network, the sensors should be put into the standby mode as long as possible. There is thus a tradeoff between the standby duration and the wakeup latency. Though the tradeoff is important, what is more important is a flexible transmission schedule that allows the tradeoff to be exploited. In this research, we use debris flow monitoring as a case study to illustrate how the tradeoff between long standby and fast wakeup can be exploited systematically with a flexible transmission schedule.

We consider a high density of sensors randomly placed in a geographical area for event monitoring. The monitoring regions of the sensors may have significant overlap, and a subset of the sensors can be turned off to conserve energy, thereby increasing the lifetime of the monitoring network. Prior work in this area does not consider the event dynamics. In this paper, we show that knowledge about the event dynamics can be exploited for significant energy savings, by putting the sensors on a periodic on/off schedule. We discuss energy-aware optimization of the periodic schedule for both cases of a synchronous and an asynchronous network. Under the periodic scheduling, coordinated sleep by the sensors can be applied orthogonally to minimize the redundancy of coverage and further improve the energy efficiency. We consider four points in the design space: synchronous periodic scheduling with and without coordinated sleep, and asynchronous periodic scheduling with and without coordinated sleep. We show that the asynchronous network exceeds the synchronous network in the coverage intensity, thereby increasing the effectiveness of the event capture, though it may also reduce the opportunities for coordinated sleep. When the sensor density is high, the asynchronous network with coordinated sleep can achieve extremely good event capture performance while being highly energy-efficient.

Sensors deployed to monitor the surrounding environment report such information as event type, location, and time when a real event of interest is detected. An adversary may identify the real event source through eavesdropping and traffic analysis. Previous work has studied the source location privacy problem under a local adversary model. In this work, we aim to provide a stronger notion: event source unobservability, which promises that a global adversary cannot know whether a real event has ever occurred even if he is capable of collecting and analyzing all the messages in the network at all the time. Clearly, event source unobservability is a desirable and critical security property for event monitoring applications, but unfortunately it is also very difficult and expensive to achieve for resource-constrained sensor network.Our main idea is to introduce carefully chosen dummy traffic to hide the real event sources in combination with mechanisms to drop dummy messages to prevent explosion of network traffic. To achieve the latter, we select some sensors as proxies that proactively filter dummy messages on their way to the base station. Since the problem of optimal proxy placement is NP-hard, we employ local search heuristics. We propose two schemes (i) Proxy-based Filtering Scheme (PFS) and (ii) Tree-based Filtering Scheme (TFS) to accurately locate proxies. Simulation results show that our schemes not only quickly find nearly optimal proxy placement, but also significantly reduce message overhead and improve message delivery ratio. A prototype of our scheme was implemented for TinyOS-based Mica2 motes.

Usage data captured and logged by computers has long been an essential source of information for software developers, support services personnel, usability designers, and learning researchers [1, 2]. Whether from mainframes, file servers, network devices, or workstations, the user event data logged in its many forms has served as an essential source of information for those who need to improve software, analyze problems, monitor security, track workflow, report on resource usage, evaluate learning activities etc. With today's generation of open and community source web-based frameworks, however, new challenges arise as to how, where, and when user activity gets captured and analyzed. These frameworks' flexibility in allowing easy integration of different applications, presentation technologies, middleware, and data sources has side effects on usage data: fragmented logs in a wide range of formats often bestrewn across many locations. This paper focuses on common issues faced especially by academic computing support personnel who need to gather and analyze user activity information within heterogeneous, distributed open source web frameworks like Sakai and uPortal. As described in this paper, these kinds of challenges can be met by drawing upon techniques for coordinated distributed event monitoring along with some basic data mining and data visualization approaches. In particular, this paper describes a work-in-progress to develop an approach towards building a distributed capture and analysis systems for a large production deployment of the Sakai Collaboration and Learning Environment in order to meet a wide range of tracking, monitoring, and reporting log analysis in one university setting.

We propose a demonstration of Cayuga, a complex event monitoring system for high speed data streams. Our demonstration will show Cayuga applied to monitoring Web feeds; the demo will illustrate the expressiveness of the Cayuga query language, the scalability of its query processing engine to high stream rates, and a visualization of the internals of the query processing engine.

Sensor networks are widely used in many applications for collecting information from the physical environment. In these applications, it is usually necessary to track the relationships between sensor data readings within a time window to detect events of interest. However, it is difficult to detect such events by using the common aggregate or selection queries. We address the problem of processing window self-join in order to detect events of interest. Self-joins are useful in tracking correlations between different sensor readings, which can indicate an event of interest. We propose the Two-Phase Self-Join (TPSJ) scheme to efficiently evaluate self-join queries for event detection in sensor networks. Our TPSJ scheme takes advantage of the properties of the events and carries out data filtering during in-network processing. We discuss TPSJ execution with one window and we extend it for continuous event monitoring. Our experimental evaluation results indicate that the TPSJ scheme is effective in reducing the amount of radio transmissions during event detection.

A distributed monitoring framework can serve as an important building block for constructing large-scale data aggregation and continuous event monitoring applications, such as IP traffic monitoring (DDoS attacks), network anomaly detection (Internet worms), accounting and bandwidth provisioning (hot spots, flash crowds), sensor monitoring and control, and grid resource monitoring. At the core of these applications is a distributed query engine that aggregates information and performs continuous tracking of queries over collections of physically-distributed and rapidly-updating data streams. The underlying aim is to provide a global view of information in the system at a reasonable cost and within a specified precision bound. To achieve this objective, a distributed monitoring system should (a) scale to a large number of streams and query attributes, (b) incur minimal communication overhead for aggregating query results, (c) be time responsive for quickly identifying anomalies, and (d) be able to bound the inaccuracy of the computed value for the aggregate function.

One of the greatest challenges in enterprises today is the lack of dynamic and ongoing information about individuals' activities, interests, and expertise. Availability of such "personal chronicles" can provide rich benefits at both an individual and enterprise level. For example, personal chronicles can help individuals to far more effectively retrieve and review their activities and interactions, while at an enterprise level they can be data-mined to identify groups of common and complementary interests and skills, or to identify implicit work processes that are commonplace in every enterprise. Today's existing tools are very limited in their support for dynamic capture of ongoing activities, in the organization and presentation of captured information, and in supporting rich annotation, search, retrieval, and publication of this information. In this paper, we propose a set of Personal Chronicling Tools (PCT) to support enterprise knowledge workers in digital event archiving and collaboration-oriented publishing. PCT is composed of four primary tools with the following capabilities: (1) event monitoring, (2) interactive annotation, (3) browse/search, and (4) edit/publish. All are designed to exploit existing enterprise infrastructure, storing captured raw data and metadata in secure databases. The first tool is a group of event monitors. These run on user client devices and capture user events such as emails, web pages browsed, instant messaging sessions, and documents edited. Monitors for new event classes are easily added as plug-ins through an XML interface. The second tool, the event annotator, enables context-sensitive user tagging and book marking of interesting moments. The third is an event browser which extends corporate email tools, providing semantic search (by embedding WordNet as a common dictionary) and the ability to follow threads of many kinds. Finally, a publishing tool facilitates the publication of relevant events with a fraction of the effort required to maintain a manual chronicle such as a weblog. This paper presents the overall system architecture, and a prototype implementation.