Search

This paper presents a long-term object tracking framework with a moving event camera under general tracking conditions. A first of its kind for these revolutionary cameras, the tracking framework uses a discriminative representation for the object with online learning, and detects and re-tracks the object when it comes back into the field-of-view. One of the key novelties is the use of an event-based local sliding window technique that tracks reliably in scenes with cluttered and textured background. In addition, Bayesian bootstrapping is used to assist real-time processing and boost the discriminative power of the object representation. On the other hand, when the object re-enters the field-of-view of the camera, a data-driven, global sliding window detector locates the object for subsequent tracking. Extensive experiments demonstrate the ability of the proposed framework to track and detect arbitrary objects of various shapes and sizes, including dynamic objects such as a human. This is a significant improvement compared to earlier works that simply track objects as long as they are visible under simpler background settings. Using the ground truth locations for five different objects under three motion settings, namely translation, rotation and 6-DOF, quantitative measurement is reported for the event-based tracking framework with critical insights on various performance issues. Finally, real-time implementation in C++ highlights tracking ability under scale, rotation, view-point and occlusion scenarios in a lab setting.

With increased focus on miniaturization and high performance in electronics, thermal management is a very important area of research today. In multiple applications such as portable electronics, consumer electronics, military applications, automobile, power electronics, high performance computing, etc. innovative thermal management strategies are necessary. In this work, two novel approaches to dissipate redundant heat better- first by novel carbonaceous-nanoparticle additives to develop thermal interface materials with superior performance and the second by using advanced metal additive manufacturing techniques to design and analyze metal-lattice based heat sinks are presented. Thermal Interface Materials with multiple carbon-based nanoparticle fillers such as coal-derived Multi Layered Graphene (MLG), standard reduced Graphene Oxide (rGO), Multi-Walled Carbon Nano Tubes (MWCNTs), and Graphene Nano-Platelets (GNPs) in thermal paste were synthesized and seen to have superior heat dissipation properties. Also, graphene was synthesized from coal through an in-house, facile, scalable and cost-effective process. The enhancement in thermal conductance varies from ~70% in the coal-MLG to ~14% in MWCNTs-based TIMs. Noteworthy is ~3.5 times larger enhancement in thermal performance with the in-house coal-derived-MLG as compared to the commercially available g-MLG. At a 3% wt. fraction of coal-MLG, enhancement in thermal conductance was almost 120% higher compared to the base thermal grease. In the second part, metal lattice-based heat sinks are designed for additive manufacturing for use in passive cooling of high-flux thermal management. A parametric optimization based on the lattice geometry, thickness, and height subject to additive manufacturing constraints is conducted. Intricate metal lattices with low mass based on the Simple Cubic, Octet, and Voronoi structures were generated by implicit modelling in nTopology® and their thermal performance was analyzed through numerical analysis using commercial CFD packages. The Voronoi lattice performed best with a significant improvement in thermal performance (~18% reduction in junction temperature difference with respect to ambient) as compared to a standard baseline Longitudinal heat Sink (LHS), while reducing the mass of the heat sink by ~2.1 times. Such optimized metal lattice-based heat sinks can lead to significant downsizing, reduction in overall mass and cost in applications where thermal management is critical with a need for low mass. We believe that such novel scalable materials and processes suited for mass production could be critical in meeting the material, design and product development needs to tackle the thermal management challenges of the future. Master of Science With increase in demand of high power and performance in electronics, there is a concurrent increase in redundant heat that needs to be dissipated. With enhanced focus and push towards electric vehicles, defense, consumer electronics, datacenter and supercomputing applications, electronics cooling is a critical area of research today. There are two primary resistances to heat- as it is removed from electronics package to the surrounding atmosphere – due to the thin layer of a material called Thermal Interface Material (TIM) at the interface between the heat sink and the package, and the resistance offered by the heat sink itself. In this work, a two-pronged approach for better cooling in electronics is presented. Firstly, carbon-based nano-sized particles are used to synthesize novel TIMs that provide superior heat transport capabilities as compared to a standard baseline. In the second approach, complex metal-lattice based heat sinks are designed for manufacturing with advanced techniques such as metal 3D printing. Multiple carbon-based nano-particle additives such as Multi Layered Graphene synthesized from coal (MLG), standard commercially available reduced Graphene Oxide (rGO), Multi-Walled Carbon Nano Tubes (MWCNTs), and Graphene Nano-Platelets (GNPs) are dispersed in thermal paste and all of the resulting composites were found to remove heat better from electronics packages. The improvement in this ability varies from ~70% in the coal-MLG to ~14% in MWCNTs-based TIMs. Noteworthy is ~3.5 times larger enhancement in the heat transport ability with the use of in-house coal-derived-MLG as compared to the commercially available g-MLG. At an 3% wt. fraction of coal-MLG, there was a 1.2x increase in thermal performance as compared to the base thermal grease. Also, it is significant to mention that MLG was synthesized from coal through an in-house, facile scalable and cost-effective process. In the second part, metal lattice-based heat sinks designed for metal 3D printing for use in passive cooling of electronics was investigated. Multiple geometric parameters such as the lattice type, thickness, and height subject to additive manufacturing constraints were studied. Intricate metal lattices with low mass based on three structures- Simple Cubic, Octet, and Voronoi were generated by implicit modelling, and their thermal performance was predicted by computer based-simulations using commercial CFD packages. The Voronoi lattice performed best with a significant reduction (~18%) in junction temperature difference with the surrounding atmosphere- as compared to a standard baseline rectangular heat sink design, while simultaneously reducing the mass of the heat sink by ~2.1 times. Such optimized metal lattice-based heat sinks can lead to significant reduction in overall mass, size, and cost in weight sensitive applications. We believe that such novel scalable materials, designs, and processes suited for mass production could be critical in meeting the material, design and product development needs to tackle the thermal management challenges of the near future.

Vehicle detection from earth-observation (EO) image has been attracting remarkable attention for its critical value in a variety of applications. Encouraged by the stunning success of deep learning techniques based on convolutional neural networks (CNNs), which have revolutionized the visual data processing community and obtained the state-of-the-art performance in a variety of classification and recognition tasks on benchmark datasets, we propose a network, called EOVNet (EO image-based vehicle detection network), to bridge the gap between the advanced deep learning research of object detection and the specific task of vehicle detection in EO images. Our network has integrated nearly all advanced techniques including very deep residual networks for feature extraction, feature pyramid to fuse multiscale features, network for proposal generation with feature sharing, and hard example mining. Moreover, our novel designs of probability-based localization and homography-based data augmentation have been investigated, resulting in further improvement of the detection performance. For performance evaluation, we have collected nearly all the representative EO datasets associated with vehicle detection. Extensive experiments on the representative datasets demonstrate that our method outperforms the state-of-the-art object detection approach Faster R-CNN++ (which is based on the Faster R-CNN framework, but with significant improvement) with 5% average precision improvement. The source code will be made available after the review process.

Most successful computer vision models transform low-level features, such as Gabor filter responses, into richer representations of intermediate or mid-level complexity for downstream visual tasks. These mid-level representations have not been explored for event cameras, although it is especially relevant to the visually sparse and often disjoint spatial information in the event stream. By making use of locally consistent intermediate representations, termed as superevents, numerous visual tasks ranging from semantic segmentation, visual tracking, depth estimation shall benefit. In essence, superevents are perceptually consistent local units that delineate parts of an object in a scene. Inspired by recent deep learning architectures, we present a novel method that employs lifetime augmentation for obtaining an event stream representation that is fed to a fully convolutional network to extract superevents. Our qualitative and quantitative experimental results on several sequences of a benchmark dataset highlights the significant potential for event-based downstream applications.

Applications in the Internet of Video Things (IoVT) domain have very tight constraints with respect to power and area. While neuromorphic vision sensors (NVS) may offer advantages over traditional imagers in this domain, the existing NVS systems either do not meet the power constraints or have not demonstrated end-to-end system performance. To address this, we improve on a recently proposed hybrid event-frame approach by using morphological image processing algorithms for region proposal and address the low-power requirement for object detection and classification by exploring various convolutional neural network (CNN) architectures. Specifically, we compare the results obtained from our object detection framework against the state-of-the-art low-power NVS surveillance system and show an improved accuracy of 82.16% from 63.1%. Moreover, we show that using multiple bits does not improve accuracy, and thus, system designers can save power and area by using only single bit event polarity information. In addition, we explore the CNN architecture space for object classification and show useful insights to trade-off accuracy for lower power using lesser memory and arithmetic operations., 4 pages, 2 figures

Efficient velocity estimation is crucial for the robust operation of navigation control loops of micro aerial vehicles (MAVs). Motivated by the research on how animals exploit their visual topographies to rapidly perform locomotion, we propose a bio-inspired method that applies quasi-parallax technique to estimate the velocity of an MAV equipped with a forward-looking stereo camera without GPS. Different to the available optical flow-based methods, our method can realize efficient metric velocity estimation without applying any depth information from either additional distance sensors or from stereopsis. In particular, the quasi-parallax technique, which claims to press maximal benefits from the configuration of two frontally parallel cameras, leverages pairs of parallel visual rays to eliminate rotational flow for translational velocity estimation, followed by refinement of the estimation of rotational velocity and translational velocity iteratively and alternately. Our method fuses the motion information from two frontal-parallel cameras without performing correspondences matching, achieving enhanced robustness and efficiency. Extensive experiments on synthesized and actual scenes demonstrate the effectiveness and efficiency of our method.

Overlap of computation and communication is critical for good application-level performance. Modern high-performance networks offer Hardware-assisted tag matching and rendezvous offload to enable communication progress without involving the host CPU. However, hardware based offload cannot be used in many situations due to various hardware limitations and performance issues. Furthermore, hardware-based designs cannot provide good overlap for common communication patterns involving unexpected messages or non-contiguous datatypes. In this paper, we address these limitations by designing a communication-aware overlap engine for MPI that uses novel hardware-assisted and software-based solutions to extract overlap for both expected and unexpected messages. The proposed design adapts to the application’s communication requirements including message size, datatype, and relative timing of processes using heuristics and history-driven predictions. We evaluate the proposed designs against state-of-the-art MPI libraries and show up to 41% and 22% reduction in latency for collective operations and stencil-based application kernels on 1024 and 128 nodes, respectively, as well as 23% improvement in communication performance of the P3DFFT application.

In this paper, we present EBBIOT-a novel paradigm for object tracking using stationary neuromorphic vision sensors in low-power sensor nodes for the Internet of Video Things (IoVT). Different from fully event based tracking or fully frame based approaches, we propose a mixed approach where we create event-based binary images (EBBI) that can use memory efficient noise filtering algorithms. We exploit the motion triggering aspect of neuromorphic sensors to generate region proposals based on event density counts with >1000X less memory and computes compared to frame based approaches. We also propose a simple overlap based tracker (OT) with prediction based handling of occlusion. Our overall approach requires 7X less memory and 3X less computations than conventional noise filtering and event based mean shift (EBMS) tracking. Finally, we show that our approach results in significantly higher precision and recall compared to EBMS approach as well as Kalman Filter tracker when evaluated over 1.1 hours of traffic recordings at two different locations., 6 pages, 5 figures

We introduce a generic visual descriptor, termed as distribution aware retinal transform (DART), that encodes the structural context using log-polar grids for event cameras. The DART descriptor is applied to four different problems, namely object classification, tracking, detection and feature matching: (1) The DART features are directly employed as local descriptors in a bag-of-features classification framework and testing is carried out on four standard event-based object datasets (N-MNIST, MNIST-DVS, CIFAR10-DVS, NCaltech-101). (2) Extending the classification system, tracking is demonstrated using two key novelties: (i) For overcoming the low-sample problem for the one-shot learning of a binary classifier, statistical bootstrapping is leveraged with online learning; (ii) To achieve tracker robustness, the scale and rotation equivariance property of the DART descriptors is exploited for the one-shot learning. (3) To solve the long-term object tracking problem, an object detector is designed using the principle of cluster majority voting. The detection scheme is then combined with the tracker to result in a high intersection-over-union score with augmented ground truth annotations on the publicly available event camera dataset. (4) Finally, the event context encoded by DART greatly simplifies the feature correspondence problem, especially for spatio-temporal slices far apart in time, which has not been explicitly tackled in the event-based vision domain., 12 pages, revision submitted to TPAMI in Nov 2018

Light detection and ranging (LiDAR) sensors have been widely deployed on intelligent systems such as unmanned ground vehicles (UGVs) and unmanned aerial vehicles (UAVs) to perform localization, obstacle detection, and navigation tasks. Thus, research into range data processing with competitive performance in terms of both accuracy and efficiency has attracted increasing attention. Sparse coding has revolutionized signal processing and led to state-of-the-art performance in a variety of applications. However, dictionary learning, which plays the central role in sparse coding techniques, is computationally demanding, resulting in its limited applicability in real-time systems. In this study, we propose sparse coding algorithms with a fixed pre-learned ridge dictionary to realize range data denoising via leveraging the regularity of laser range measurements in man-made environments. Experiments on both synthesized data and real data demonstrate that our method obtains accuracy comparable to that of sophisticated sparse coding methods, but with much higher computational efficiency.

This paper describes the development of a novel algorithm to tackle the problem of real-time video stabilization for unmanned aerial vehicles (UAVs). There are two main components in the algorithm: (1) By designing a suitable model for the global motion of UAV, the proposed algorithm avoids the necessity of estimating the most general motion model, projective transformation, and considers simpler motion models, such as rigid transformation and similarity transformation. (2) To achieve a high processing speed, optical-flow based tracking is employed in lieu of conventional tracking and matching methods used by state-of-the-art algorithms. These two new ideas resulted in a real-time stabilization algorithm, developed over two phases. Stage I considers processing the whole sequence of frames in the video while achieving an average processing speed of 50fps on several publicly available benchmark videos. Next, Stage II undertakes the task of real-time video stabilization using a multi-threading implementation of the algorithm designed in Stage I., Comment: Journal Paper

Parallel programmers face the often irreconcilable goals of programmability and performance. HPC systems use distributed memory for scalability, thereby sacrificing the programmability advantages of shared memory programming models. Furthermore, the rapid adoption of heterogeneous architectures, often with non-cache-coherent memory systems, has further increased the challenge of supporting shared memory programming models. Our primary objective is to define a memory consistency model that presents the familiar thread-based shared memory programming model, but allows good application performance on non-cache-coherent systems, including distributed memory clusters and accelerator-based systems. We propose regional consistency (RegC), a new consistency model that achieves this objective. Results on up to 256 processors for representative benchmarks demonstrate the potential of RegC in the context of our prototype distributed shared memory system., Comment: 8 pages, 7 figures, 1 table; as submitted to CCGRID 2013