Search

Sensors of various modalities and capabilities, especially cameras, have become ubiquitous in our environment. Their intended use is wide ranging and encompasses surveillance, transportation, entertainment, education, healthcare, emergency response, disaster recovery, and the like. Technological advances and the low cost of such sensors enable deployment of large-scale camera networks in large metropolises such as London and New York. Multimedia algorithms for analyzing and drawing inferences from video and audio have also matured tremendously in recent times. Despite all these advances, large-scale reliable systems for media-rich sensor-based applications, often classified as situation-awareness applications, are yet to become commonplace. Why is that? There are several forces at work here. First, the system abstractions are just not at the right level for quickly prototyping such applications on a large scale. Second, while Moore's law has held true for predicting the growth of processing power, the volume of data that applications are called upon to handle is growing similarly, if not faster. Enormous amount of sensing data is continually generated for real-time analysis in such applications. Further, due to the very nature of the application domain, there are dynamic and demanding resource requirements for such analyses. The lack of right set of abstractions for programing such applications coupled with their data-intensive nature have hitherto made realizing reliable large-scale situation-awareness applications difficult. Incidentally, situation awareness is a very popular but ill-defined research area that has attracted researchers from many different fields. In this paper, we adopt a strong systems perspective and consider the components that are essential in realizing a fully functional situation-awareness system.

Cameras are becoming ubiquitous. Applications including video-based surveillance and emergency response exploit camera networks to detect anomalies in real time and reduce collateral damage. A well-known technique for detecting anomalies is spatio-temporal analysis -- an inferencing technique employed by domain experts (e.g., vision researchers) to answer spatio-temporal queries. In this paper, we propose a distributed framework that facilitates the development and deployment of spatio-temporal analysis applications on large-scale camera networks and backend computing resources. We make the following contributions: (a) an investigation of the computation/communication costs associated with spatio-temporal analysis, (b) a programming framework designed for large-scale spatio-temporal analysis, and (c) performance evaluations for each step of the spatio-temporal analysis with realistic algorithms.

The ubiquitous deployment of mobile and sensor devices is creating a new environment, namely the Internet of Things(IoT), that enables a wide range of future Internet applications. In this work, we present Mobile Fog, a high level programming model for the future Internet applications that are geospatially distributed, large-scale, and latency-sensitive. We analyze use cases for the programming model with camera network and connected vehicle applications to show the efficacy of Mobile Fog. We also evaluate application performance through simulation.

We introduce a novel abstraction, the target container (TC), which serves as a parallel programming model and execution framework for developing complex applications for tracking multiple targets in a large-scale camera network. The key insight is to allow the domain expert (e.g., a vision researcher) to focus on the algorithmic details of target tracking and let the system deal with providing the computational resources (cameras, networking, and processing) to enable target tracking. Each TC has a one-to-one correspondence with a target, possibly tracked from multiple cameras. The domain expert provides the code modules for target tracking (such as detectors and trackers) as handlers to the TC system. The handlers are invoked dynamically by the TC system to discover new targets (detector) and to follow existing targets (tracker). The TC system also provides an interface for merging TCs whenever they are determined to be corresponding to the same target. This paper presents the design of the TC system, details of an experimental prototype, and an example application to demonstrate the simplicity of using the TC programming model.