Search

In the field of machine learning (ML) for materials optimization, active learning algorithms, such as Bayesian Optimization (BO), have been leveraged for guiding autonomous and high-throughput experimentation systems. However, very few studies have evaluated the efficiency of BO as a general optimization algorithm across a broad range of experimental materials science domains. In this work, we evaluate the performance of BO algorithms with a collection of surrogate model and acquisition function pairs across five diverse experimental materials systems, namely carbon nanotube polymer blends, silver nanoparticles, lead-halide perovskites, as well as additively manufactured polymer structures and shapes. By defining acceleration and enhancement metrics for general materials optimization objectives, we find that for surrogate model selection, Gaussian Process (GP) with anisotropic kernels (automatic relevance detection, ARD) and Random Forests (RF) have comparable performance and both outperform the commonly used GP without ARD. We discuss the implicit distributional assumptions of RF and GP, and the benefits of using GP with anisotropic kernels in detail. We provide practical insights for experimentalists on surrogate model selection of BO during materials optimization campaigns.

Articulated motion analysis often utilizes strong prior knowledge such as a known or trained parts model for humans. Yet, the world contains a variety of articulating objects--mammals, insects, mechanized structures--where the number and configuration of parts for a particular object is unknown in advance. Here, we relax such strong assumptions via an unsupervised, Bayesian nonparametric parts model that infers an unknown number of parts with motions coupled by a body dynamic and parameterized by SE(D), the Lie group of rigid transformations. We derive an inference procedure that utilizes short observation sequences (image, depth, point cloud or mesh) of an object in motion without need for markers or learned body models. Efficient Gibbs decompositions for inference over distributions on SE(D) demonstrate robust part decompositions of moving objects under both 3D and 2D observation models. The inferred representation permits novel analysis, such as object segmentation by relative part motion, and transfers to new observations of the same object type.

© 2018 IEEE. When performing multi-robot tasks, sensory feedback is crucial in reducing uncertainty for correct execution. Yet the utilization of sensors should be planned as an integral part of the task planning, taken into account several factors such as the tolerance of different inferred properties of the scene and interaction with different agents. In this paper we handle this complex problem in a principled, yet efficient way. We use surrogate predictors based on open-loop simulation to estimate and bound the probability of success for specific tasks. We reason about such task-specific uncertainty approximants and their effectiveness. We show how they can be incorporated into a multi-robot planner, and demonstrate results with a team of robots performing assembly tasks.

© 2017 IEEE. Point cloud alignment is a common problem in computer vision and robotics, with applications ranging from 3D object recognition to reconstruction. We propose a novel approach to the alignment problem that utilizes Bayesian non-parametrics to describe the point cloud and surface normal densities, and branch and bound (BB) optimization to recover the relative transformation. BB uses a novel, refinable, near-uniform tessellation of rotation space using 4D tetrahedra, leading to more efficient optimization compared to the common axis-angle tessellation. We provide objective function bounds for pruning given the proposed tessellation, and prove that BB converges to the optimum of the cost function along with providing its computational complexity. Finally, we empirically demonstrate the efficiency of the proposed approach as well as its robustness to real-world conditions such as missing data and partial overlap.

The advent of widely available photo collections covering broad geographic areas has spurred significant advances in large-scale urban scene modeling. While much emphasis has been placed on reconstruction and visualization, the utility of such models extends well beyond. Specifically, these models should support a wide variety of reasoning tasks (or queries), and thus enable advanced scene study. Driven by this interest, we analyze 3D representations for their utility to perform queries. Since representations as well as queries are highly heterogeneous, we build on a categorization that serves as a coupling interface between both domains. Equipped with our taxonomy and the notion of uncertainty in the representation, we quantify the utility of representations for solving three archetypal reasoning tasks in terms of accuracy, uncertainty and computational complexity. We provide an empirical analysis of these intertwined realms on challenging real and synthetic urban scenes.

Standard image based segmentation approaches perform poorly when there is little or no contrast along boundaries of different regions. In such cases, segmentation is largely performed manually using prior knowledge of the shape and relative location of the underlying structures combined with partially discernible boundaries. We present an automated approach guided by covariant shape deformations of neighboring structures, which is an additional source of prior information. Captured by a shape atlas, these deformations are transformed into a statistical model using the logistic function. Structure boundaries, anatomical labels, and image inhomogeneities are estimated simultaneously within an Expectation-Maximization formulation of the maximum a posteriori probability estimation problem. We demonstrate the approach on 20 brain magnetic resonance images showing superior performance, particularly in cases where purely image based methods fail.

Objects and structures within man-made environments typically exhibit a high degree of organization in the form of orthogonal and parallel planes. Traditional approaches utilize these regularities via the restrictive, and rather local, Manhattan World (MW) assumption which posits that every plane is perpendicular to one of the axes of a single coordinate system. The aforementioned regularities are especially evident in the surface normal distribution of a scene where they manifest as orthogonally-coupled clusters. This motivates the introduction of the Manhattan-Frame (MF) model which captures the notion of an MW in the surface normals space, the unit sphere, and two probabilistic MF models over this space. First, for a single MF we propose novel real-time MAP inference algorithms, evaluate their performance and their use in drift-free rotation estimation. Second, to capture the complexity of real-world scenes at a global scale, we extend the MF model to a probabilistic mixture of Manhattan Frames (MMF). For MMF inference we propose a simple MAP inference algorithm and an adaptive Markov-Chain Monte-Carlo sampling algorithm with Metropolis-Hastings split/merge moves that let us infer the unknown number of mixture components. We demonstrate the versatility of the MMF model and inference algorithm across several scales of man-made environments., United States. Office of Naval Research. Multidisciplinary University Research Initiative (Award N00014- 11-1-06)

Sensor planning and active sensing, long studied in robotics, adapt sensor parameters to maximize a utility function while constraining resource expenditures. Here we consider information gain as the utility function. While these concepts are often used to reason about 3D sensors, these are usually treated as a predefined, black-box, component. In this paper we show how the same principles can be used as part of the 3D sensor. We describe the relevant generative model for structured-light 3D scanning and show how adaptive pattern selection can maximize information gain in an open-loop-feedback manner. We then demonstrate how different choices of relevant variable sets (corresponding to the subproblems of locatization and mapping) lead to different criteria for pattern selection and can be computed in an online fashion. We show results for both subproblems with several pattern dictionary choices and demonstrate their usefulness for pose estimation and depth acquisition., United States. Office of Naval Research (Grant N00014-09-1-1051), United States. Army Research Office (Grant W911NF-11- 1-0391), United States. Office of Naval Research (Grant N00014- 11-1-0688)

We consider a methodology for integrating multiple sensors along with semantic information to enhance scene representations. We propose a probabilistic generative model for inferring semantically-informed aerial reconstructions from multi-modal data within a consistent mathematical framework. The approach, called Semantically-Aware Aerial Reconstruction (SAAR), not only exploits inferred scene geometry, appearance, and semantic observations to obtain a meaningful categorization of the data, but also extends previously proposed methods by imposing structure on the prior over geometry, appearance, and semantic labels. This leads to more accurate reconstructions and the ability to fill in missing contextual labels via joint sensor and semantic information. We introduce a new multi-modal synthetic dataset in order to provide quantitative performance analysis. Additionally, we apply the model to real-world data and exploit OpenStreetMap as a source of semantic observations. We show quantitative improvements in reconstruction accuracy of large-scale urban scenes from the combination of LiDAR, aerial photography, and semantic data. Furthermore, we demonstrate the model's ability to fill in for missing sensed data, leading to more interpretable reconstructions.

Sparse graphical models have proven to be a flexible class of multivariate probability models for approximating high-dimensional distributions. In this paper, we propose techniques to exploit this modeling ability for binary classification by discriminatively learning such models from labeled training data, i.e., using both positive and negative samples to optimize for the structures of the two models. We motivate why it is difficult to adapt existing generative methods, and propose an alternative method consisting of two parts. First, we develop a novel method to learn tree-structured graphical models which optimizes an approximation of the log-likelihood ratio. We also formulate a joint objective to learn a nested sequence of optimal forests-structured models. Second, we construct a classifier by using ideas from boosting to learn a set of discriminative trees. The final classifier can interpreted as a likelihood ratio test between two models with a larger set of pairwise features. We use cross-validation to determine the optimal number of edges in the final model. The algorithm presented in this paper also provides a method to identify a subset of the edges that are most salient for discrimination. Experiments show that the proposed procedure outperforms generative methods such as Tree Augmented Naive Bayes and Chow-Liu as well as their boosted counterparts.

We consider the problem of detecting and localizing a material release utilizing sparse sensor measurements and formulate the problem as one of abrupt change detection. Methods which rely on single-sensor detection require dense deployment to achieve adequate coverage; costly sensors preclude such approaches. Furthermore, localization requires the fusion of multiple sensor measurements. Fusion in sparse sensor configurations is dependent on the knowledge of the dynamics of particle dispersion, which is, itself, problematic due to the inherent randomness on the wind field. We consider the efficacy of using an approximate dynamic model with coarse parameter estimates for the detection and localization of material releases. Specifically, we consider propagation models consisting of diffusion plus transport according to a Gaussian dispersion model. Assuming a known wind field, unconstrained intersensor communication, and a centralized processor, we derive optimal inference algorithms and provide a hybrid detection-localization hypothesis-testing framework with linear growth in the hypothesis space. We then analyze the probability of detection, time-to-detection, and localization performance as a function of the number of sensors. Furthermore, we examine the impact on performance when the underlying dynamical model deviates from the assumed model. This detailed analysis provides the basis for the design of more sophisticated algorithms for 1) performing robust detection followed by refined nonlinear parameter estimation which provides enhanced localization, and 2) distributed architectures aimed at conserving communication resources in which detections within local clusters are used to trigger more intensive intercluster communication to improve detection and localization

The objective of this research program was to develop mathematical foundations of information gathering through an integrated theory of sensing, inference, and control. The goal of the team was to develop a new framework for autonomous operations that will extend the state of the art in distributed learning and modeling from data, and tightly integrate these models into new decentralized cooperative planning algorithms. The main output of this effort will be a fundamental theory to integrate decentralized information driven planning methods for heterogenous teams with nonparametric Bayesian models of uncertainty. The feasibility and aspects of the value of the theory were demonstrated via integrated software and hardware experiments.

Superpixel segmentation is a key step in many image processing and vision tasks. Our recently-proposed connectivity-constrained probabilistic model [1] yields high-quality super-pixels. Seemingly, however, connectivity constraints preclude parallelized inference. As such, the implementation from [1] is serial. The contributions of this work are as follows. First, we demonstrate that effective parallelization is possible via a fast GPU implementation that scales gracefully with both the number of pixels and number of superpixels. Second, we show that the superpixels are improved by replacing the fixed and restricted spatial covariances from [1] with a flexible Bayesian prior. Quantitative evaluation on public benchmarks shows the proposed method outperforms the state-of-the-art. We make our implementation publicly available.

Drift of the rotation estimate is a well known problem in visual odometry systems as it is the main source of positioning inaccuracy. We propose three novel algorithms to estimate the full 3D rotation to the surrounding Manhattan World (MW) in as short as 20 ms using surface-normals derived from the depth channel of a RGB-D camera. Importantly, this rotation estimate acts as a structure compass which can be used to estimate the bias of an odometry system, such as an inertial measurement unit (IMU), and thus remove its angular drift. We evaluate the run-time as well as the accuracy of the proposed algorithms on groundtruth data. They achieve zerodrift rotation estimation with RMSEs below 3.4° by themselves and below 2.8° when integrated with an IMU in a standard extended Kalman filter (EKF). Additional qualitative results show the accuracy in a large scale indoor environment as well as the ability to handle fast motion. Selected segmentations of scenes from the NYU depth dataset demonstrate the robustness of the inference algorithms to clutter and hint at the usefulness of the segmentation for further processing., United States. Office of Naval Research. Multidisciplinary University Research Initiative6 (Awards N00014-11-1-0688 and N00014-10-1-0936), National Science Foundation (U.S.) (Award IIS-1318392)

Structural regularities in man-made environments reflect in the distribution of their surface normals. Describing these surface normal distributions is important in many computer vision applications, such as scene understanding, plane segmentation, and regularization of 3D reconstructions. Based on the small-variance limit of Bayesian nonparametric von-Mises-Fisher (vMF) mixture distributions, we propose two new flexible and efficient k-means-like clustering algorithms for directional data such as surface normals. The first, DP-vMF-means, is a batch clustering algorithm derived from the Dirichlet process (DP) vMF mixture. Recognizing the sequential nature of data collection in many applications, we extend this algorithm to DDP-vMF-means, which infers temporally evolving cluster structure from streaming data. Both algorithms naturally respect the geometry of directional data, which lies on the unit sphere. We demonstrate their performance on synthetic directional data and real 3D surface normals from RGB-D sensors. While our experiments focus on 3D data, both algorithms generalize to high dimensional directional data such as protein backbone configurations and semantic word vectors., United States. Office of Naval Research. Multidisciplinary University Research Initiative (Grant N00014- 11-1-0688), United States. Army Research Office. Multidisciplinary University Research Initiative (Grant W911NF-11-1-0391)

In continuously operating robotic systems, efficient representation of the previously seen camera feed is crucial. Using a highly efficient compression coreset method, we formulate a new method for hierarchical retrieval of frames from large video streams collected online by a moving robot. We demonstrate how to utilize the resulting structure for efficient loop-closure by a novel sampling approach that is adaptive to the structure of the video. The same structure also allows us to create a highly-effective search tool for large-scale videos, which we demonstrate in this paper. We show the efficiency of proposed approaches for retrieval and loop closure on standard datasets, and on a large-scale video from a mobile camera., Hon Hai/Foxconn International Holdings Ltd., Lincoln Laboratory, Singapore-MIT Alliance for Research and Technology Center (Future Urban Mobility Project), United States. Office of Naval Research. Multidisciplinary University Research Initiative (Grant N00014-09-1-1051)

We analyze the complexity of evaluating information rewards for measurement selection in sparse graphical models under the assumption that measurements are drawn from a limited number of nodes subject to a finite budget. Previous analyses [1, 2, 3] exploit the submodular property of conditional mutual information to demonstrate that greedy measurement selection come with near-optimal guarantees As noted in [4] typical formulations assume oracle value models. However, [1, 2, 5] allude to a more significant source of complexity, namely computing the measurement reward. Here, we focus on Gaussian models and show that by exploiting sparsity in the measurement model, the complexity of planning is substantially reduced. We also demonstrate that by utilizing the information form additional significant reductions in complexity may be realized.

We propose novel finite-dimensional spaces of Rn ? Rn transformations, n ? {1, 2, 3}, derived from (continuously-defined) parametric stationary velocity fields. Particularly, we obtain these transformations, which are diffeomorphisms, by fast and highly-accurate integration of continuous piecewise-affine velocity fields, we also provide an exact solution for n = 1. The simple-yet-highly-expressive proposed representation handles optional constraints (e.g., volume preservation) easily and supports convenient modeling choices and rapid likelihood evaluations (facilitating tractable inference over latent transformations). Its applications include, but are not limited to: unconstrained optimization over monotonic functions, modeling cumulative distribution functions or histograms, time warping, image registration, landmark-based warping, real-time diffeomorphic image editing. Our code is available at https://github.com/freifeld/cpabDiffeo.

A statistical model is presented that combines the registration of an atlas with the segmentation of magnetic resonance images. We use an Expectation Maximization-based algorithm to find a solution within the model, which simultaneously estimates image artifacts, anatomical labelmaps, and a structure-dependent hierarchical mapping from the atlas to the image space. The algorithm produces segmentations for brain tissues as well as their substructures. We demonstrate the approach on a set of 22 magnetic resonance images. On this set of images, the new approach performs significantly better than similar methods which sequentially apply registration and segmentation.

Determining the structure of dependencies among a set of variables is a common task in many signal and image processing applications, including multitarget tracking and computer vision. In this paper, we present an information-theoretic, machine learning approach to problems of this type. We cast this problem as a hypothesis test between factorizations of variables into mutually independent subsets. We show that the likelihood ratio can be written as sums of two sets of Kullback-Leibler (KL) divergence terms. The first set captures the structure of the statistical dependencies within each hypothesis, whereas the second set measures the details of model differences between hypotheses. We then consider the case when the signal prior models are unknown, so that the distributions of interest must be estimated directly from data, showing that the second set of terms is (asymptotically) negligible and quantifying the loss in hypothesis separability when the models are completely unknown. We demonstrate the utility of nonparametric estimation methods for such problems, providing a general framework for determining and distinguishing between dependency structures in highly uncertain environments. Additionally, we develop a machine learning approach for estimating lower bounds on KL divergence and mutual information from samples of high-dimensional random variables for which direct density estimation is infeasible. We present empirical results in the context of three prototypical applications: association of signals generated by sources possessing harmonic behavior, scene correspondence using video imagery, and detection of coherent behavior among sets of moving objects.

Audio and visual signals arriving from a common source are detected using a signal-level fusion technique. A probabilistic multimodal generation model is introduced and used to derive an information theoretic measure of cross-modal correspondence. Nonparametric statistical density modeling techniques can characterize the mutual information between signals from different domains. By comparing the mutual information between different pairs of signals, it is possible to identify which person is speaking a given utterance and discount errant motion or audio from other utterances or nonspeech events.

Scattering from man-made objects in SAR imagery exhibits aspect and frequency dependencies which are not well modeled by standard SAR imaging techniques. If ignored, these deviations will reduce recognition performance due to the model mismatch, but when appropriately accounted for, these deviations from the ideal point scattering model can be exploited as attributes to better distinguish scatterers and their respective targets. With this premise in mind, we have developed an efficient modeling framework that incorporates scatterer anisotropy. One of the products of our analysis is the assignment of an anisotropy label to each scatterer conveying the degree of anisotropy. Anisotropic behavior is commonly predicted for geometric scatterers (scatterers with a simple geometric structure), but it may also arise from volumetric scatterers (random arrangements of interfering point scatterers). Analysis of anisotropy arising from these two modalities shows a clear source-dependent relationship between the anisotropy classification and parameters of the scatterer. In particular, the degree of anisotropy is closely related to the size of the scatterer, and increasing the aperture size reduces the incidence of volumetric anisotropy but preserves the detection rate for geometric anisotropy. This result helps to address the question in the SAR community regarding the utility of wide-aperture SAR data for ATR since wide-aperture data reveals geometric anisotropy while resolving volumetric anisotropy into individual isotropic scatterers.

The appeal of distributed sensing and computation is matched by the formidable challenges it presents in terms of estimation and communication. Applications range from military surveillance to collaborative office environments. Despite the attractiveness of exploiting networks of low-power and low-cost sensors, how to do so is a difficult problem. In this paper, we adopt a statistical viewpoint of such networks, and identify three key challenges. The first is to develop principled methods for low-level fusion of sensors measuring different modalities. We discuss an information-theoretic approach to sensor fusion, and present experimental results using audio and video data. The core component of this method is the learning of a nonparametric joint statistical model for the sensing modes. Secondly, we discuss how one might apply such a sensor fusion algorithm to acquire the relative geometry of a network of sensors using passively-sensed data. Specifically, we show how the fusion method previously developed can be used to find correspondences between pairs of long-baseline sensors. Finding such correspondences is, in general, the starting point for recovering the geometry. Finally, we discuss two iterative algorithms for performing inference on graphical models with cycles. Such models provide a flexible framework for constructing globally consistent statistical models from a set of local interactions. Importantly, the algorithms that we present allow information to be transmitted and processed in a distributed manner.

We propose an integrated probabilistic model for multi-modal fusion of aerial imagery, LiDAR data, and (optional) GPS measurements. The model allows for analysis and dense reconstruction (in terms of both geometry and appearance) of large 3D scenes. An advantage of the approach is that it explicitly models uncertainty and allows for missing data. As compared with image-based methods, dense reconstructions of complex urban scenes are feasible with fewer observations. Moreover, the proposed model allows one to estimate absolute scale and orientation and reason about other aspects of the scene, e.g., detection of moving objects. As formulated, the model lends itself to massively-parallel computing. We exploit this in an efficient inference scheme that utilizes both general purpose and domain-specific hardware components. We demonstrate results on large-scale reconstruction of urban terrain from LiDAR and aerial photography data.

Objects and structures within man-made environments typically exhibit a high degree of organization in the form of orthogonal and parallel planes. Traditional approaches to scene representation exploit this phenomenon via the somewhat restrictive assumption that every plane is perpendicular to one of the axes of a single coordinate system. Known as the Manhattan-World model, this assumption is widely used in computer vision and robotics. The complexity of many real-world scenes, however, necessitates a more flexible model. We propose a novel probabilistic model that describes the world as a mixture of Manhattan frames: each frame defines a different orthogonal coordinate system. This results in a more expressive model that still exploits the orthogonality constraints. We propose an adaptive Markov-Chain Monte-Carlo sampling algorithm with Metropolis-Hastings split/merge moves that utilizes the geometry of the unit sphere. We demonstrate the versatility of our Mixture-of-Manhattan-Frames model by describing complex scenes using depth images of indoor scenes as well as aerial-LiDAR measurements of an urban center. Additionally, we show that the model lends itself to focal-length calibration of depth cameras and to plane segmentation., United States. Office of Naval Research. Multidisciplinary University Research Initiative (Award N00014-11-1-0688), United States. Defense Advanced Research Projects Agency (Award FA8650-11-1-7154), Technion, Israel Institute of Technology (MIT Postdoctoral Fellowship Program)

We present an integrated probabilistic model for layered object tracking that combines dynamics on implicit shape representations, topological shape constraints, adaptive appearance models, and layered flow. The generative model combines the evolution of appearances and layer shapes with a Gaussian process flow and explicit layer ordering. Efficient MCMC sampling algorithms are developed to enable a particle filtering approach while reasoning about the distribution of object boundaries in video. We demonstrate the utility of the proposed tracking algorithm on a wide variety of video sources while achieving state-of-the-art results on a boundary-accurate tracking dataset.

This paper discusses a framework for learning based on information theoretic criteria. A novel algorithm based on Renyi's quadratic entropy is used to train, directly from a data set, linear or nonlinear mappers for entropy maximization or minimization. We provide an intriguing analogy between the computation and an information potential measuring the interactions among the data samples. We also propose two approximations to the Kulback-Leibler divergence based on quadratic distances (Cauchy-Schwartz inequality and Euclidean distance). These distances can still be computed using the information potential. We test the newly proposed distances in blind source separation (unsupervised learning) and in feature extraction for classification (supervised learning). In blind source separation our algorithm is capable of separating instantaneously mixed sources, and for classification the performance of our classifier is comparable to the support vector machines (SVMs).

We develop a generative probabilistic model for temporally consistent super pixels in video sequences. In contrast to supermodel methods, object parts in different frames are tracked by the same temporal super pixel. We explicitly model flow between frames with a bilateral Gaussian process and use this information to propagate super pixels in an online fashion. We consider four novel metrics to quantify performance of a temporal super pixel representation and demonstrate superior performance when compared to supermodel methods.

The minimum average correlation energy (MACE) filter, which is linear and shift invariant, has been used extensively in the area of automatic target detection and recognition (ATD/R). We present a nonlinear extension of the MACE filter, based on a statistical formulation of the optimization criterion, of which the linear MACE filter is a special case. A method by which nonlinear topologies can be incorporated into the filter design is presented and adaptation issues are discussed. In particular, we outline a method by which training exhaustively over the image plane is avoided which leads to much shorter adaptation. Experimental results, using target chips from 35 GHz TABILS 24 inverse synthetic aperture (ISAR) data are presented and performance comparisons are made between the MACE filter and this nonlinear extension.

We present a new generative image model, integrating techniques arising from two different domains: manifold modeling and Markov random fields. First, we develop a probabilistic model with a mixture of hyperplanes to approximate the manifold of orientable image patches, and demonstrate that it is more effective than the field of experts in expressing local texture patterns. Next, we develop a construction that yields an MRF for coherent image generation, given a configuration of local patch models, and thereby establish a prior distribution over an MRF space. Taking advantage of the model structure, we derive a variational inference algorithm, and apply it to low-level vision. In contrast to previous methods that rely on a single MRF, the method infers an approximate posterior distribution of MRFs, and recovers the underlying images by combining the predictions in a Bayesian fashion. Experiments quantitatively demonstrate superior performance as compared to state-of-the-art methods on image denoising and inpainting.

Markov random fields play a central role in solving a variety of low level vision problems, including denoising, in-painting, segmentation, and motion estimation. Much previous work was based on MRFs with hand-crafted networks, yet the underlying graphical structure is rarely explored. In this paper, we show that if appropriately estimated, the MRF's graphical structure, which captures significant information about appearance and motion, can provide crucial guidance to low level vision tasks. Motivated by this observation, we propose a principled framework to solve low level vision tasks via an exponential family of MRFs with variable structures, which we call Switchable MRFs. The approach explicitly seeks a structure that optimally adapts to the image or video along the pursuit of task-specific goals. Through theoretical analysis and experimental study, we demonstrate that the proposed method addresses a number of drawbacks suffered by previous methods, including failure to capture heavy-tail statistics, computational difficulties, and lack of generality.

Many vision problems, such as object recognition and image synthesis, are greatly impacted by deformation of objects. In this paper, we develop a deformation model based on Lie algebraic analysis. This work aims to provide a generative model that explicitly decouples deformation from appearance, which is fundamentally different from the prior work that focuses on deformation-resilient features or metrics. Specifically, the deformation group for each object can be characterized by a set of Lie algebraic basis. Such basis for different objects are related via parallel transport. Exploiting the parallel transport relations, we formulate an optimization problem, and derive an algorithm that jointly estimates the deformation basis for a class of objects, given a set of images resulted from the action of the deformations. We test the proposed model empirically on both character recognition and face synthesis.

We propose a principled framework to model persistent motion in dynamic scenes. In contrast to previous efforts on object tracking and optical flow estimation that focus on local motion, we primarily aim at inferring a global model of persistent and collective dynamics. With this in mind, we first introduce the concept of geometric flow that describes motion simultaneously over space and time, and derive a vector space representation based on Lie algebra. We then extend it to model complex motion by combining multiple flows in a geometrically consistent manner. Taking advantage of the linear nature of this representation, we formulate a stochastic flow model, and incorporate a Gaussian process to capture the spatial coherence more effectively. This model leads to an efficient and robust algorithm that can integrate both point pairs and frame differences in motion estimation. We conducted experiments on different types of videos. The results clearly demonstrate that the proposed approach is effective in modeling persistent motion.

We present a novel representation for modeling textured regions subject to smooth variations in orientation and scale. Utilizing the steerable pyramid of Simoncelli and Freeman as a basis, we decompose textured regions of natural images into explicit local attributes of contrast, bias, scale, and orientation. Additionally, we impose smoothness on these attributes via Markov random fields. The combination allows for demonstrable improvements in common scene analysis applications including unsupervised segmentation, reflectance and shading estimation, and estimation of the radiometric response function from a single image.

Fusion of 3D laser radar (LIDAR) imagery and aerial optical imagery is an efficient method for constructing 3D virtual reality models. One difficult aspect of creating such models is registering the optical image with the LIDAR point cloud, which is characterized as a camera pose estimation problem. We propose a novel application of mutual information registration methods, which exploits the statistical dependency in urban scenes of optical appearance with measured LIDAR elevation. We utilize the well known downhill simplex optimization to infer camera pose parameters. We discuss three methods for measuring mutual information between LIDAR imagery and optical imagery. Utilization of OpenGL and graphics hardware in the optimization process yields registration times dramatically lower than previous methods. Using an initial registration comparable to GPS/INS accuracy, we demonstrate the utility of our algorithm with a collection of urban images and present 3D models created with the fused imagery.

We present a novel method for modeling dynamic visual phenomena, which consists of two key aspects. First, the integral motion of constituent elements in a dynamic scene is captured by a common underlying geometric transform process. Second, a Lie algebraic representation of the transform process is introduced, which maps the transformation group to a vector space, and thus overcomes the difficulties due to the group structure. Consequently, the statistical learning techniques based on vector spaces can be readily applied. Moreover, we discuss the intrinsic connections between the Lie algebra and the Linear dynamical processes, showing that our model induces spatially varying fields that can be estimated from local motions without continuous tracking. Following this, we further develop a statistical framework to robustly learn the flow models from noisy and partially corrupted observations. The proposed methodology is demonstrated on real world phenomenon, inferring common motion patterns from surveillance videos of crowded scenes and satellite data of weather evolution.

Information measures have long been studied in the context of hypothesis testing leading to variety of bounds on performance based on the information content of a signal or the divergence between distributions. Here we consider the problem of estimation of information content for high-dimensional signals for purposes of classification. Direct estimation of information for high-dimensional signals is generally not tractable therefore we consider an extension to a method first suggested in [1] in which high dimensional signals are mapped to lower dimensional feature spaces yielding lower bounds on information content. We develop an affine-invariant gradient method and examine the utility of the resulting estimates for predicting classification performance empirically.

This paper explores modeling the dependency structure among multiple vector time-series. We focus on a large classes of structures which yield efficient and tractable exact inference. Specifically, we use directed trees and forests to model causal interactions among time-series. These models are incorporated in a dynamic setting in which a latent variable indexes evolving structures. We demonstrate the utility of the method by analyzing the interaction of multiple moving objects.

Sparse signal representations and approximations from overcomplete dictionaries have become an invaluable tool recently. In this paper, we develop a new, heuristic, graph-structured, sparse signal representation algorithm for overcomplete dictionaries that can be decomposed into subdictionaries and whose dictionary elements can be arranged in a hierarchy. Around this algorithm, we construct a methodology for advanced image formation in wide-angle synthetic aperture radar (SAR), defining an approach for joint anisotropy characterization and image formation. Additionally, we develop a coordinate descent method for jointly optimizing a parameterized dictionary and recovering a sparse representation using that dictionary. The motivation is to characterize a phenomenon in wide-angle SAR that has not been given much attention before: migratory scattering centers, i.e., scatterers whose apparent spatial location depends on aspect angle. Finally, we address the topic of recovering solutions that are sparse in more than one objective domain by introducing a suitable sparsifying cost function. We encode geometric objectives into SAR image formation through sparsity in two domains, including the normal parameter space of the Hough transform.

We present a method for sequential learning of increasingly complex graphical models for discriminating between two hypotheses. We generate forests for each hypothesis, each with no more edges than a spanning tree, which optimize an information-theoretic criteria. The method relies on a straightforward extension of the efficient max-weight spanning tree (MWST) algorithm by incorporating multivalued edge-weights. Each iteration produces nested forests with increasing number of edges; each provably optimal as compared to alternative forests. Empirical results demonstrate superior probability of error as compared to generative approaches.

We present an algorithm to generate samples from probability distributions on the space of curves. We view a traditional curve evolution energy functional as a negative log probability distribution and sample from it using a Markov chain Monte Carlo (MCMC) algorithm. We define a proposal distribution by generating smooth perturbations to the normal of the curve and show how to compute the transition probabilities to ensure that the samples come from the posterior distribution. We demonstrate some advantages of sampling methods such as robustness to local minima, better characterization of multi-modal distributions, access to some measures of estimation error, and ability to easily incorporate constraints on the curve.

The logarithm of the odds ratio (LogOdds) is frequently used in areas such as artificial neural networks, economics, and biology, as an alternative representation of probabilities. Here, we use LogOdds to place probabilistic atlases in a linear vector space. This representation has several useful properties for medical imaging. For example, it not only encodes the shape of multiple anatomical structures but also captures some information concerning uncertainty. We demonstrate that the resulting vector space operations of addition and scalar multiplication have natural probabilistic interpretations. We discuss several examples for placing label maps into the space of LogOdds. First, we relate signed distance maps, a widely used implicit shape representation, to LogOdds and compare it to an alternative that is based on smoothing by spatial Gaussians. We find that the LogOdds approach better preserves shapes in a complex multiple object setting. In the second example, we capture the uncertainty of boundary locations by mapping multiple label maps of the same object into the LogOdds space. Third, we define a framework for non-convex interpolations among atlases that capture different time points in the aging process of a population. We evaluate the accuracy of our representation by generating a deformable shape atlas that captures the variations of anatomical shapes across a population. The deformable atlas is the result of a principal component analysis within the LogOdds space. This atlas is integrated into an existing segmentation approach for MR images. We compare the performance of the resulting implementation in segmenting 20 test cases to a similar approach that uses a more standard shape model that is based on signed distance maps. On this data set, the Bayesian classification model with our new representation outperformed the other approaches in segmenting subcortical structures.

We consider the problem of jointly forming images and determining anisotropy from wide-angle synthetic aperture radar (SAR) measurements. Conventional SAR image formation techniques assume isotropic scattering, which is not valid with wide-angle apertures. We present a method based on a sparse representation of aspect-dependent scattering with an overcomplete dictionary composed of elements with varying levels of angular persistence. Solved as an inverse problem, the result is a complex-valued, aspect-dependent response for each spatial location in a scene. Our formulation leads to an optimization problem for which we develop a tractable, graph-structured approximate algorithm. We present experimental results on realistic electromagnetic simulations demonstrating the effectiveness of the proposed approach.

We present a novel level-set method for evolving open surfaces embedded in three-dimensional volumes. We adapt the method for statistical detection and segmentation of cytoarchitectonic regions of the cortical ribbon (e.g., Brodmann areas). In addition, we incorporate an explicit interface appearance model which is oriented normal to the open surface, allowing one to model characteristics beyond voxel intensities and high gradients. We show that such models are well suited to detecting embedded cortical structures. Appearance models of the interface are used in two ways: firstly, to evolve an open surface in the normal direction for the purpose of detecting the location of the surface, and secondly, to evolve the boundary of the surface in a direction tangential to the surface in order to delineate the extent of a specific Brodmann area within the cortical ribbon. The utility of the method is demonstrated on a challenging ex-vivo structural MR dataset for detection of Brodmann area 17.

Understanding the dependency structure of a set of variables is a key component in various signal processing applications which involve data association. The simple task of detecting whether any dependency exists is particularly difficult when models of the data are unknown or difficult to characterize because of high-dimensional measurements. We review the use of nonparametric tests for characterizing dependency and how to carry out these tests with high-dimensional observations. In addition we present a method to assess the significance of the tests.

The concept of the Logarithm of the Odds (LogOdds) is frequently used in areas such as artificial neural networks, economics, and biology. Here, we utilize LogOdds for a shape representation that demonstrates desirable properties for medical imaging. For example, the representation encodes the shape of an anatomical structure as well as the variations within that structure. These variations are embedded in a vector space that relates to a probabilistic model. We apply our representation to a voxel based segmentation algorithm. We do so by embedding the manifold of Signed Distance Maps (SDM) into the linear space of LogOdds. The LogOdds variant is superior to the SDM model in an experiment segmenting 20 subjects into subcortical structures. We also use LogOdds in the non-convex interpolation between space conditioned distributions. We apply this model to a longitudinal schizophrenia study using quadratic splines. The resulting time-continuous simulation of the schizophrenic aging process has a higher accuracy then a model based on convex interpolation.