Search

The retina and primary visual cortex (V1) both exhibit diverse neural populations sensitive to diverse visual features. Yet it remains unclear how neural populations in each area partition stimulus space to span these features. One possibility is that neural populations are organized into discrete groups of neurons, with each group signaling a particular constellation of features. Alternatively, neurons could be continuously distributed across feature-encoding space. To distinguish these possibilities, we presented a battery of visual stimuli to the mouse retina and V1 while measuring neural responses with multi-electrode arrays. Using machine learning approaches, we developed a manifold embedding technique that captures how neural populations partition feature space and how visual responses correlate with physiological and anatomical properties of individual neurons. We show that retinal populations discretely encode features, while V1 populations provide a more continuous representation. Applying the same analysis approach to convolutional neural networks that model visual processing, we demonstrate that they partition features much more similarly to the retina, indicating they are more like big retinas than little brains. [ABSTRACT FROM AUTHOR]

Invoking the manifold assumption in machine learning requires knowledge of the manifold's geometry and dimension, and theory dictates how many samples are required. However, in most applications, the data are limited, sampling may not be uniform, and the manifold's properties are unknown; this implies that neighborhoods must adapt to the local structure. We introduce an algorithm for inferring adaptive neighborhoods for data given by a similarity kernel. Starting with a locally conservative neighborhood (Gabriel) graph, we sparsify it iteratively according to a weighted counterpart. In each step, a linear program yields minimal neighborhoods globally, and a volumetric statistic reveals neighbor outliers likely to violate manifold geometry. We apply our adaptive neighborhoods to nonlinear dimensionality reduction, geodesic computation, and dimension estimation. A comparison against standard algorithms using, for example, k -nearest neighbors, demonstrates the usefulness of our approach. [ABSTRACT FROM AUTHOR]

Greedy basis pursuit (GBP) is introduced for computing sparse signal representations by using overcomplete dictionaries. It is shown that GBP can provide a faster and better alternative to standard linear programming approaches to basis pursuit.

Hierarchical image structures are abundant in computer vision and have been used to encode part structure, scale spaces, and a variety of multiresolution features. In this paper, we describe a framework for indexing such representations that embeds the topological structure of a directed acyclic graph (DAG) into a low-dimensional vector space. Based on a novel spectral characterization of a DAG, this topological signature allows us to efficiently retrieve a promising set of candidates from a database of models using a simple nearest-neighbor search. We establish the insensitivity of the signature to minor perturbation of graph structure due to noise, occlusion, or node split/merge. To accommodate large-scale occlusion, the DAG rooted at each nonleaf node of the query "votes" for model objects that share that "part," effectively accumulating local evidence in a model DAG's topological subspaces. We demonstrate the approach with a series of indexing experiments in the domain of view-based 3D object recognition using shock graphs. Index Terms--Structural indexing, graph spectra, object recognition, shock graphs.

While complaints about typical edge operators are common, proposals articulating a notion of the "perfect" edge map are comparatively rare, hindering the improvement of contour enhancement techniques. To address this situation, we suggest that one objective of visual contour computation is the estimation of a clean sketch from a corrupted rendition, the latter modeling noisy and low contrast edge or line operator responses to an image. Our formal model of this clean sketch is the curve indicator random field (CIRF), whose role is to provide a basis for defining edge likelihood models by eliminating the parameter along each curve to create an image of curves. For curves modeled with stationary Markov processes, this ideal edge prior is non-Gaussian and its moment generating functional has a form closely related to the Feynman-Kac formula. This sketch model leads to a nonlinear, minimum mean squared error contour enhancement filter that requires the solution of two elliptic partial differential equations. The framework is also independent of the order of the contour model, allowing us to introduce a Markov process model for contour curvature. We analyze the distribution of such curves and show that its mode is the Euler spiral, a curve minimizing changes in curvature. Example computations using the contour enhancement filter with the curvature-based contour model are provided, highlighting how the filter is curvature-selective even when curvature is absent in the input. Index Terms--Random fields, Markov processes, Feynman-Kac formula, curvature Brownian motion, edge detection, posterior mean, sketch, contour enhancement, curve, elastica, Euler spiral, orientation, direction.

Locally parallel dense patterns--sometimes called texture flows--define a perceptually coherent structure of particular significance to perceptual organization. We argue that with applications ranging from image segmentation and edge classification to shading analysis and shape interpretation, texture flows deserve attention equal to edge segment grouping and curve completion. This paper develops the notion of texture flow from a geometrical point of view to argue that local measurements of such structures must incorporate two curvatures. We show how basic theoretical considerations lead to a unique model for the local behavior of the flow and to a notion of texture flow "good continuation." This, in turn, translates to a specification of consistency constraints between nearby flow measurements which we use for the computation of globally (piecewise) coherent structure through the contextual framework of relaxation labeling. We demonstrate the results on synthetic and natural images. Index Terms--Texture flow, perceptual organization, social conformity of a line, good continuation, texture segmentation, line discontinuities, point singularities, shading flow, local parallelism, orientation diffusion, tangential curvature, normal curvature, relaxation labeling.

The standard approach to edge detection is based on a model of edges as large step changes in intensity. This approach fails to reliably detect and localize edges in natural images where blur scale and contrast can vary over a broad range. The main problem is that the appropriate spatial scale for local estimation depends upon the local structure of the edge, and thus varies unpredictably over the image. Here we show that knowledge of sensor properties and operator norms can be exploited to define a unique, locally computable minimum reliable scale for local estimation at each point in the image. This method for local scale control is applied to the problem of detecting and localizing edges in images with shallow depth of field and shadows. We show that edges spanning a broad range of blur scales and contrasts can be recovered accurately by a single system with no input parameters other than the second moment of the sensor noise. A natural dividend of this approach is a measure of the thickness of contours which can be used to estimate focal and penumbral blur. Local scale control is shown to be important for the estimation of blur in complex images, where the potential for interference between nearby edges of very different blur scale requires that estimates be made at the minimum reliable scale. Index Terms - Edge detection, localization, scale space, blur estimation, defocus, shadows.

Several active contour models have been proposed which merge the curve evolution framework with classical energy minimization methods for segmentation. The Euclidean curve shortening equation which defines the gradient direction where the Euclidean perimeter is rapidly shrinking motivates the models. The Euclidean arc-length must be multiplied by a function designed to fit the features of interest in the intensity image.

It is well-known that the problem of matching two relational structures can be posed as an equivalent problem of finding a maximal clique in a (derived) "association graph." However, it is not clear how to apply this approach to computer vision problems where the graphs are hierarchically organized, i.e., are trees, since maximal cliques are not constrained to preserve the partial order. Here, we provide a solution to the problem of matching two trees by constructing the association graph using the graph-theoretic concept of connectivity. We prove that, in the new formulation, there is a one-to-one correspondence between maximal cliques and maximal subtree isomorphisms. This allows us to cast the tree matching problem as an indefinite quadratic program using the Motzkin-Straus theorem, and we use "replicator" dynamical systems developed in theoretical biology to solve it. Such continuous solutions to discrete problems are attractive because they can motivate analog and biological implementations. The framework is also extended to the matching of attributed trees by using weighted association graphs. We illustrate the power of the approach by matching articulated and deformed shapes described by shock trees. Index Terms--Maximal subtree isomorphisms, association graphs, maximal cliques, replicator dynamical systems, shock trees, shape recognition.

We exploit a key result from visual psychophysics-that individuals perceive shape qualitatively -- to develop the use of a geometrical-topological -invariant" (the Morse{Smale complex) relating image structure with surface structure. Differences across individuals are minimal near certain configurations such as ridges and boundaries, and it is these configurations that are often represented in line drawings. In particular, we introduce a method for inferring a qualitative three-dimensional shape from shading patterns that link the shape-from-shading inference with shape-from-contour inference. For a given shape, certain shading patches approach -line drawings" in a well-defined limit. Under this limit, and invariably with respect to rendering choices, these shading patterns provide a qualitative description of the surface. We further show that, under this model, the contours partition the surface into meaningful parts using the Morse-Smale complex. These critical contours are the (perceptually) stable parts of this complex and are invariant over a wide class of rendering models. Intuitively, our main result shows that critical contours partition smooth surfaces into bumps and valleys, in effect providing a scaffold on the image from which a full surface can be interpolated. [ABSTRACT FROM AUTHOR]

Many organisms navigate gradients by alternating straight motions (runs) with random reorientations (tumbles), transiently suppressing tumbles whenever attractant signal increases. This induces a functional coupling between movement and sensation, since tumbling probability is controlled by the internal state of the organism which, in turn, depends on previous signal levels. Although a negative feedback tends to maintain this internal state close to adapted levels, positive feedback can arise when motion up the gradient reduces tumbling probability, further boosting drift up the gradient. Importantly, such positive feedback can drive large fluctuations in the internal state, complicating analytical approaches. Previous studies focused on what happens when the negative feedback dominates the dynamics. By contrast, we show here that there is a large portion of physiologically-relevant parameter space where the positive feedback can dominate, even when gradients are relatively shallow. We demonstrate how large transients emerge because of non-normal dynamics (non-orthogonal eigenvectors near a stable fixed point) inherent in the positive feedback, and further identify a fundamental nonlinearity that strongly amplifies their effect. Most importantly, this amplification is asymmetric, elongating runs in favorable directions and abbreviating others. The result is a “ratchet-like” gradient climbing behavior with drift speeds that can approach half the maximum run speed of the organism. Our results thus show that the classical drawback of run-and-tumble navigation—wasteful runs in the wrong direction—can be mitigated by exploiting the non-normal dynamics implicit in the run-and-tumble strategy. [ABSTRACT FROM AUTHOR]

Vision problems are inherently ambiguous: Do abrupt brightness changes correspond to object boundaries? Are smooth intensity changes due to shading or material properties? For stereo: Which point in the left image corresponds to which point in the right one? What is the role of color in visual information processing? To answer these (seemingly different) questions we develop an analogy between the role of orientation in organizing visual cortex and tangents in differential geometry. Machine learning experiments suggest using geometry as a surrogate for high-order statistical interactions. The cortical columnar architecture becomes a bundle structure in geometry. Connection forms within these bundles suggest answers to the above questions, and curvatures emerge in key roles. More generally, our path through these questions suggests an overall strategy for solving the inverse problems of vision: decompose the global problems into networks of smaller ones and then seek constraints from these coupled problems to reduce ambiguity. Neural computations thus amount to satisfying constraints rather than seeking uniform approximations. Even when no global formulation exists one may be able to find localized structures on which ambiguity is minimal; these can then anchor an overall approximation. [ABSTRACT FROM PUBLISHER]

Shape-from-shading (SFS) is a classical inverse problem in computer and human vision. This shape reconstruction problem is inherently ill-posed. We show that the isophotes on smooth surfaces with Lambertian reflectance can be directly related to surface properties without consideration of the light source. Using techniques from modern differential geometry, we derive relationships between the curvature of the isophotes and the shape operator for the surface. Neurobiology motivates the geometric approach, and our calculations allow us to characterize the matching local family of surfaces that can result from any given shading patch. We illustrate the local ambiguity in several examples. [ABSTRACT FROM AUTHOR]

Many traditional two-view stereo algorithms explicitly or implicitly use the frontal parallel plane assumption when exploiting contextual information since, e.g., the smoothness prior biases toward constant disparity (depth) over a neighborhood. This introduces systematic errors to the matching process for slanted or curved surfaces. These errors are nonnegligible for detailed geometric modeling of natural objects such as a human face. We show how to use contextual information geometrically to avoid such errors. A differential geometric study of smooth surfaces allows contextual information to be encoded in Cartan's moving frame model over local quadratic approximations, providing a framework of geometric consistency for both depth and surface normals; the accuracy of our reconstructions argues for the sufficiency of the approximation. In effect, Cartan's model provides the additional constraint necessary to move beyond the frontal parallel plane assumption in stereo reconstruction. It also suggests how geometry can extend surfaces to account for unmatched points due to partial occlusion. [ABSTRACT FROM AUTHOR]

Data fusion is a natural and common approach to recovering the state of physical systems. But the dissimilar appearance of different sensors remains a fundamental obstacle. We propose a unified embedding scheme for multisensory data, based on the spectral diffusion framework, which addresses this issue. Our scheme is purely data-driven and assumes no a priori statistical or deterministic models of the data sources. To extract the underlying structure, we first embed separately each input channel; the resultant structures are then combined in diffusion coordinates. In particular, as different sensors sample similar phenomena with different sampling densities, we apply the density invariant Laplace-Beltrami embedding. This is a fundamental issue in multisensor acquisition and processing, overlooked in prior approaches. We extend previous work on group recognition and suggest a novel approach to the selection of diffusion coordinates. To verify our approach, we demonstrate performance improvements in audio/visual speech recognition. [ABSTRACT FROM AUTHOR]

Standard approaches to stereo correspondence have difficulty when scene structure does not lie in or near the frontal parallel plane, in part because an orientation disparity as well as a positional disparity is introduced. We propose a correspondence algorithm based on differential geometry, that takes explicit advantage of both disparities. The algorithm relates the 2D differential structure (position, tangent, and curvature) of curves in the left and right images to the Frenet approximation of the (3D) space curve. A compatibility function is defined via transport of the Frenet frames, and they are matched by relaxing this compatibility function on overlapping neighborhoods along the curve. The remaining false matches are concurrently eliminated by a model of “near” and “far” neurons derived from neurobiology. Examples on scenes with complex 3D structures are provided. [ABSTRACT FROM AUTHOR]