Search

This paper proposes a new method of calculating a matching metric for motion estimation. The proposed method splits the information in the source images into multiple scale and orientation subbands, reduces the subband values to a binary representation via an adaptive thresholding algorithm, and uses mutual information to model the similarity of corresponding square windows in each image. A moving window strategy is applied to recover a dense estimated motion field whose properties are explored. The proposed matching metric is a sum of mutual information scores across space, scale, and orientation. This facilitates the exploitation of information diversity in the source images. Experimental comparisons are performed amongst several related approaches, revealing that the proposed matching metric is better able to exploit information diversity, generating more accurate motion fields. [ABSTRACT FROM AUTHOR]

The great variations of videographic skills in videography, camera designs, compression and processing protocols, communication and bandwidth environments, and displays leads to an enormous variety of video impairments. Current no-reference (NR) video quality models are unable to handle this diversity of distortions. This is true in part because available video quality assessment databases contain very limited content, fixed resolutions, were captured using a small number of camera devices by a few videographers and have been subjected to a modest number of distortions. As such, these databases fail to adequately represent real world videos, which contain very different kinds of content obtained under highly diverse imaging conditions and are subject to authentic, complex, and often commingled distortions that are difficult or impossible to simulate. As a result, NR video quality predictors tested on real-world video data often perform poorly. Toward advancing NR video quality prediction, we have constructed a large-scale video quality assessment database containing 585 videos of unique content, captured by a large number of users, with wide ranges of levels of complex, authentic distortions. We collected a large number of subjective video quality scores via crowdsourcing. A total of 4776 unique participants took part in the study, yielding over 205 000 opinion scores, resulting in an average of 240 recorded human opinions per video. We demonstrate the value of the new resource, which we call the live video quality challenge database (LIVE-VQC), by conducting a comparison with leading NR video quality predictors on it. This paper is the largest video quality assessment study ever conducted along several key dimensions: number of unique contents, capture devices, distortion types and combinations of distortions, study participants, and recorded subjective scores. The database is available for download on this link: http://live.ece.utexas.edu/research/LIVEVQC/index.html. [ABSTRACT FROM AUTHOR]

This paper investigates the use of a qualitative inverse scattering method known as the linear sampling method (LSM) for imaging underwater scenes using limited aperture receiver configurations. The LSM is based on solving a set of unstable integral equations known as the far-field equations and whose stability breaks down even further for under-sampled observation aperture data. Based on the results of a recent study concerning multi-frequency LSM imaging, we propose an iterative inversion method that is founded upon a compressive sensing framework. In particular, we leverage multi-frequency diversity in the data by imposing a partial frequency variation prior on the solution which we show is justified when the frequency bandwidth is sampled finely enough. We formulate an alternating direction method of multiplier approach to minimize the proposed cost function. Proof of concept is established through numerically generated data as well as experimental acoustic measurements taken in a shallow pool facility at the U.S Naval Research Laboratory. [ABSTRACT FROM AUTHOR]

In this paper, we propose an extremely efficient algorithm for visual re-ranking. By considering the original pairwise distance in the contextual space, we develop a feature vector called sparse contextual activation (SCA) that encodes the local distribution of an image. Hence, re-ranking task can be simply accomplished by vector comparison under the generalized Jaccard metric, which has its theoretical meaning in the fuzzy set theory. In order to improve the time efficiency of re-ranking procedure, inverted index is successfully introduced to speed up the computation of generalized Jaccard metric. As a result, the average time cost of re-ranking for a certain query can be controlled within 1 ms. Furthermore, inspired by query expansion, we also develop an additional method called local consistency enhancement on the proposed SCA to improve the retrieval performance in an unsupervised manner. On the other hand, the retrieval performance using a single feature may not be satisfactory enough, which inspires us to fuse multiple complementary features for accurate retrieval. Based on SCA, a robust feature fusion algorithm is exploited that also preserves the characteristic of high time efficiency. We assess our proposed method in various visual re-ranking tasks. Experimental results on Princeton shape benchmark (3D object), WM-SRHEC07 (3D competition), YAEL data set B (face), MPEG-7 data set (shape), and Ukbench data set (image) manifest the effectiveness and efficiency of SCA. [ABSTRACT FROM AUTHOR]

There exist a significant number of benchmarks for evaluating the performance of boundary detection algorithms, most of them relying on some sort of comparison of the automatically-generated boundaries with human-labeled ones. Such benchmarks are composed of a representative image data set, as well as a comparison measure on the universe of boundary images. Despite many such data sets and measures have been proposed, there is no clear way of knowing which combinations of them are the most suitable for the task. In this paper, we introduce four criteria that allow for a sensible evaluation of the performance of a comparison measure on a given data set. The criteria mimic the way in which humans understand boundary images, as well as their ability to recognize the underlying scenes. These criteria can, as a final goal, quantify the ability of the boundary detection benchmarks to evaluate the performance of boundary detection methods, either edge-based or segmentation-based. [ABSTRACT FROM AUTHOR]

In this paper, we present a novel algorithm to simultaneously accomplish color quantization and dithering of images. This is achieved by minimizing a perception-based cost function, which considers pixel-wise differences between filtered versions of the quantized image and the input image. We use edge aware filters in defining the cost function to avoid mixing colors on the opposite sides of an edge. The importance of each pixel is weighted according to its saliency. To rapidly minimize the cost function, we use a modified multi-scale iterative conditional mode (ICM) algorithm, which updates one pixel a time while keeping other pixels unchanged. As ICM is a local method, careful initialization is required to prevent termination at a local minimum far from the global one. To address this problem, we initialize ICM with a palette generated by a modified median-cut method. Compared with previous approaches, our method can produce high-quality results with a fewer visual artifacts but also requires significantly less computational effort. [ABSTRACT FROM AUTHOR]

In this paper, we address the problem of imaging membrane proteins for single-particle cryo-electron microscopy reconstruction of the isolated protein structure. More precisely, we propose a method for learning and removing the interfering vesicle signals from the micrograph, prior to reconstruction. In our approach, we estimate the subspace of the vesicle structures and project the micrographs onto the orthogonal complement of this subspace. We construct a 2D statistical model of the vesicle structure, based on higher order singular value decomposition (HOSVD), by considering the structural symmetries of the vesicles in the polar coordinate plane. We then propose to lift the HOSVD model to a novel hierarchical model by summarizing the multidimensional HOSVD coefficients by their principal components. Along with the model, a solid vesicle normalization scheme and model selection criterion are proposed to make a compact and general model. The results show that the vesicle structures are accurately separated from the background by the HOSVD model that is also able to adapt to the asymmetries of the vesicles. This is a promising result and suggests even wider applicability of the proposed approach in learning and removal of statistical structures. [ABSTRACT FROM PUBLISHER]

Dilated convolutions support expanding receptive field without parameter exploration or resolution loss, which turn out to be suitable for pixel-level prediction problems. In this paper, we propose multiscale single image super-resolution (SR) based on dilated convolutions. We adopt dilated convolutions to expand the receptive field size without incurring additional computational complexity. We mix standard convolutions and dilated convolutions in each layer, called mixed convolutions, i.e., in the mixed convolutional layer, and the feature extracted by dilated convolutions and standard convolutions are concatenated. We theoretically analyze the receptive field and intensity of mixed convolutions to discover their role in SR. Mixed convolutions remove blind spots and capture the correlation between low-resolution (LR) and high-resolution (HR) image pairs successfully, thus achieving good generalization ability. We verify those properties of mixed convolutions by training 5-layer and 10-layer networks. We also train a 20-layer deep network to compare the performance of the proposed method with those of the state-of-the-art ones. Moreover, we jointly learn maps with different scales from a LR image to its HR one in a single network. Experimental results demonstrate that the proposed method outperforms the state-of-the-art ones in terms of PSNR and SSIM, especially for a large-scale factor. [ABSTRACT FROM AUTHOR]

In the current scenario of high-resolution inverse synthetic aperture radar (ISAR) imaging, the non-cooperative targets may have strong maneuverability, which tends to cause time-variant Doppler modulation and imaging plane in the echoed data. Furthermore, it is still a challenge to realize ISAR imaging of maneuvering targets from sparse aperture (SA) data. In this paper, we focus on the problem of 3D geometry and motion estimations of maneuvering targets for interferometric ISAR (InISAR) with SA. For a target of uniformly accelerated rotation, the rotational modulation in echo is formulated as chirp sensing code under a chirp-Fourier dictionary to represent the maneuverability. In particular, a joint multi-channel imaging approach is developed to incorporate the multi-channel data and treat the multi-channel ISAR image formation as a joint-sparsity constraint optimization. Then, a modified orthogonal matching pursuit (OMP) algorithm is employed to solve the optimization problem to produce high-resolution range-Doppler (RD) images and chirp parameter estimation. The 3D target geometry and the motion estimations are followed by using the acquired RD images and chirp parameters. Herein, a joint estimation approach of 3D geometry and rotation motion is presented to realize outlier removing and error reduction. In comparison with independent single-channel processing, the proposed joint multi-channel imaging approach performs better in 2D imaging, 3D imaging, and motion estimation. Finally, experiments using both simulated and measured data are performed to confirm the effectiveness of the proposed algorithm. [ABSTRACT FROM AUTHOR]

Super resolution (SR) for real-life video sequences is a challenging problem due to complex nature of the motion fields. In this paper, a novel blind SR method is proposed to improve the spatial resolution of video sequences, while the overall point spread function of the imaging system, motion fields, and noise statistics are unknown. To estimate the blur(s), first, a nonuniform interpolation SR method is utilized to upsample the frames, and then, the blur(s) is(are) estimated through a multi-scale process. The blur estimation process is initially performed on a few emphasized edges and gradually on more edges as the iterations continue. Also for faster convergence, the blur is estimated in the filter domain rather than the pixel domain. The high-resolution frames are estimated using a cost function that has the fidelity and regularization terms of type Huber–Markov random field to preserve edges and fine details. The fidelity term is adaptively weighted at each iteration using a masking operation to suppress artifacts due to inaccurate motions. Very promising results are obtained for real-life videos containing detailed structures, complex motions, fast-moving objects, deformable regions, or severe brightness changes. The proposed method outperforms the state of the art in all performed experiments through both subjective and objective evaluations. The results are available online at http://lyle.smu.edu/~rajand/Video_SR/. [ABSTRACT FROM AUTHOR]

We present a deep learning strategy to fuse multiple semantic cues for complex event recognition. In particular, we tackle the recognition task by answering how to jointly analyze human actions (who is doing what), objects (what), and scenes (where). First, each type of semantic features (e.g., human action trajectories) is fed into a corresponding multi-layer feature abstraction pathway, followed by a fusion layer connecting all the different pathways. Second, the correlations of how the semantic cues interacting with each other are learned in an unsupervised cross-modality autoencoder fashion. Finally, by fine-tuning a large-margin objective deployed on this deep architecture, we are able to answer the question on how the semantic cues of who, what, and where compose a complex event. As compared with the traditional feature fusion methods (e.g., various early or late strategies), our method jointly learns the essential higher level features that are most effective for fusion and recognition. We perform extensive experiments on two real-world complex event video benchmarks, MED’11 and CCV, and demonstrate that our method outperforms the best published results by 21% and 11%, respectively, on an event recognition task. [ABSTRACT FROM AUTHOR]

Higher Order Aberrations (HOAs) are complex refractive errors in the human eye that cannot be corrected by regular lens systems. Researchers have developed numerous approaches to analyze the effect of these refractive errors; the most popular among these approaches use Zernike polynomial approximation to describe the shape of the wavefront of light exiting the pupil after it has been altered by the refractive errors. We use this wavefront shape to create a linear imaging system that simulates how the eye perceives source images at the retina. With phase information from this system, we create a second linear imaging system to modify source images so that they would be perceived by the retina without distortion. By modifying source images, the visual process cascades two optical systems before the light reaches the retina, a technique that counteracts the effect of the refractive errors. While our method effectively compensates for distortions induced by HOAs, it also introduces blurring and loss of contrast; a problem that we address with Total Variation Regularization. With this technique, we optimize source images so that they are perceived at the retina as close as possible to the original source image. To measure the effectiveness of our methods, we compute the Euclidean error between the source images and the images perceived at the retina. When comparing our results with existing corrective methods that use deconvolution and total variation regularization, we achieve an average of 50% reduction in error with lower computational costs. [ABSTRACT FROM AUTHOR]

Person re-identification is a fundamental task in many computer vision and image understanding systems. Due to appearance variations from different camera views, person re-identification still poses an important challenge. In the literature, KISSME has already been introduced as an effective distance metric learning method using pairwise constraints to improve the re-identification performance. Computationally, it only requires two inverse covariance matrix estimations. However, the linear transformation induced by KISSME is not powerful enough for more complex problems. We show that KISSME can be kernelized, resulting in a nonlinear transformation, which is suitable for many real-world applications. Moreover, the proposed kernel method can be used for learning distance metrics from structured objects without having a vectorial representation. The effectiveness of our method is validated on five publicly available data sets. To further apply the proposed kernel method efficiently when data are collected sequentially, we introduce a fast incremental version that learns a dissimilarity function in the feature space without estimating the inverse covariance matrices. The experiments show that the latter variant can obtain competitive results in a computationally efficient manner. [ABSTRACT FROM AUTHOR]

Presents the table of contents for this issue of the publication. [ABSTRACT FROM PUBLISHER]