Search

We propose the fast optical flow extractor, a filtering method that recovers artifact-free optical flow fields from HEVC-compressed video. To extract accurate optical flow fields, we form a regularized optimization problem that considers the smoothness of the solution and the pixelwise confidence weights of an artifact- ridden HEVC motion field. Solving such an optimization problem is slow, so we first convert the problem into a confidence-weighted filtering task. By leveraging the already-available HEVC motion parameters, we achieve a 100-fold speed-up in the running times compared to similar methods, while producing subpixel-accurate flow estimates. The fast optical flow extractor is useful when video frames are already available in coded formats. Our method is not specific to a coder, and works with motion fields from video coders such as H.264/AVC and HEVC. [ABSTRACT FROM AUTHOR]

In this paper, an original observation model for multiresolution optical flow estimation is introduced. Multiresolution frameworks, often based on coarse-to-fine warping strategies, are widely used by state-of-the-art optical flow methods. They allow the recovery of large motions by successive estimations of the flow field at several resolution levels. Although such approaches perform very efficiently and usually lead to faster minimizations, they generally consider independent problems at each resolution levels and do not exploit the existing interactions between scales (especially the influences of fine scales on larger ones). In this paper, we tackle this issue by proposing a flexible framework, inspired from fluid mechanics, able to partly counter these limitations. For each resolution level, our process filters the equations of interest and decomposes the key variables into resolved (i.e., at a given resolution) and unresolved (i.e., at finer resolutions) components. This enables to derive a new data term that takes into account, at coarse resolutions, the influence of their unresolved parts. From this new term, we propose two different estimation strategies, depending on whether we explicitly know the type of relations between the different scales (as for physical processes) or not. In order to test the efficiency of this new observation model, we have embedded it in a simple multiresolution Lucas-Kanade estimator. Comparing the usual optical flow constraint equation with this new term in the same motion estimation procedure, it clearly appears that the proposed term leads to more consistent estimates and prevents from errors propagation apparition during the estimation. In all situations (synthetic, real, physical images or not), our new term is able to greatly improve the results compared with usual conservation constraints. [ABSTRACT FROM PUBLISHER]

In this paper, a new low-complexity true-motion estimation (TME) algorithm is proposed for video processing applications, such as motion-compensated temporal frame interpolation (MCTFI) or motion-compensated frame rate up-conversion (MCFRUC). Regular motion estimation, which is often used in video coding, aims to find the motion vectors (MVs) to reduce the temporal redundancy, whereas TME aims to track the projected object motion as closely as possible. TME is obtained by imposing implicit and/or explicit smoothness constraints on the block-matching algorithm. To produce better quality-interpolated frames, the dense motion field at interpolation time is obtained for both forward and backward MVs; then, bidirectional motion compensation using forward and backward MVs is applied by mixing both elegantly. Finally, the performance of the proposed algorithm for MCTFI is demonstrated against recently proposed methods and smoothness constraint optical flow employed by a professional video production suite. Experimental results show that the quality of the interpolated frames using the proposed method is better when compared with the MCFRUC techniques. [ABSTRACT FROM AUTHOR]

This paper proposes the use of an adaptive locally oriented coordinate frame when calculating an optical flow field. The coordinate frame is aligned with the least curvature direction in a local window about each pixel. This has advantages to both fitting the flow field to the image data and in imposing smoothness constraints between neighboring pixels. In terms of fitting, robustness is obtained to a wider variety of image motions due to the extra invariance provided by the coordinate frame. Smoothness constraints are naturally propagated along image boundaries which often correspond to motion boundaries. In addition, moving objects can be efficiently segmented in the least curvature direction. We show experimentally the benefits of the method and demonstrate robustness to fast rotational motion, such as what often occurs in human motion. [ABSTRACT FROM AUTHOR]

We present a novel method to perform an accurate registration of 3-D nonrigid bodies by using phase-shift properties of the dual-tree complex wavelet transform (\DT-\BBC\WT). Since the phases of \DT-\BBC\WT coefficients change approximately linearly with the amount of feature displacement in the spatial domain, motion can be estimated using the phase information from these coefficients. The motion estimation is performed iteratively: first by using coarser level complex coefficients to determine large motion components and then by employing finer level coefficients to refine the motion field. We use a parametric affine model to describe the motion, where the affine parameters are found locally by substituting into an optical flow model and by solving the resulting overdetermined set of equations. From the estimated affine parameters, the motion field between the sensed and the reference data sets can be generated, and the sensed data set then can be shifted and interpolated spatially to align with the reference data set. [ABSTRACT FROM PUBLISHER]

We propose a fast and robust 2D-affine global motion estimation algorithm based on phase-correlation in the Fourier-Mellin domain and robust least square model fitting of sparse motion vector field and its application for digital image stabilization. Rotation-scale-translation (RST) approximation of affine parameters is obtained at the coarsest level of the image pyramid, thus ensuring convergence for a much larger range of motions. Despite working at the coarsest resolution level, using subpixel-accurate phase correlation provides sufficiently accurate coarse estimates for the subsequent refinement stage of the algorithm. The refinement stage consists of RANSAC based robust least-square model fitting for sparse motion vector field, estimated using block-based subpixel-accurate phase correlation at randomly selected high activity regions in finest level of image pyramid. Resulting algorithm is very robust to outliers such as foreground objects and flat regions. We investigate the robustness of the proposed method for digital image stabilization application. Experimental results show that the proposed algorithm is capable of estimating larger range of motions as compared to another phase correlation method and optical flow algorithm. [ABSTRACT FROM AUTHOR]

Fluid motion estimation from time-sequenced images is a significant image analysis task. Its application is widespread in experimental fluidics research and many related areas like biomedical engineering and atmospheric sciences. In this paper, we present a novel flow computation framework to estimate the flow velocity vectors from two consecutive image frames. In an energy minimization-based flow computation, we propose a novel data fidelity term, which: 1) can accommodate various measures, such as cross-correlation or sum of absolute or squared differences of pixel intensities between image patches; 2) has a global mechanism to control the adverse effect of outliers arising out of motion discontinuities, proximity of image borders; and 3) can go hand-in-hand with various spatial smoothness terms. Further, the proposed data term and related regularization schemes are both applicable to dense and sparse flow vector estimations. We validate these claims by numerical experiments on benchmark flow data sets. [ABSTRACT FROM AUTHOR]

We propose a temporal modeling approach for determining image motion from a sequence of images wherein the inherent motion is periodic over time. To exploit the periodic nature of the motion, we use a Fourier harmonic representation model the temporal evolution of the motion field for the entire sequence. We then determine the motion field simultaneously for the different image frames by estimating the parameters of this representation model, where the model order in the Fourier representation serves as a regularization parameter on the temporal coherence of the motion field. This approach can take advantage of the statistics of all the available data in the image sequence. In our experiments, we tested the proposed approach on several motion types at different noise levels, including translational motion, convergent/ divergent motion, and cardiac motion. Our results demonstrate that this approach could lead to more robust estimation the motion field in the presence of strong imaging noise compared to a frame-by-frame estimation approach. [ABSTRACT FROM AUTHOR]

Optical flow can be reliably estimated between areas visible in two images, but not in occlusion areas. If optical flow is needed in the whole image domain, one approach is to use additional views of the same scene. If such views are unavailable, an often-used alternative is to extrapolate optical flow in occlusion areas. Since the location of such areas is usually unknown prior to optical flow estimation, this is usually performed in three steps. First, occlusion-ignorant optical flow is estimated, then occlusion areas are identified using the estimated (unreliable) optical flow, and, finally, the optical flow is corrected using the computed occlusion areas. This approach, however, does not permit interaction between optical flow and occlusion estimates. In this paper, we permit such interaction by proposing a variational formulation that jointly computes optical flow, implicitly detects occlusions and extrapolates optical flow in occlusion areas. The extrapolation mechanism is based on anisotropic diffusion and uses the underlying image gradient to preserve structure, such as optical flow discontinuities. Our results show significant improvements in the computed optical flow fields over other approaches, both qualitatively and quantitatively. [ABSTRACT FROM AUTHOR]

In this paper, we address some of the major issues in optical flow within a new framework assuming nonstationary statistics for the motion field and for the errors. Problems addressed include the preservation of discontinuities, model/data errors, outliers, confidence measures, and performance evaluation. In solving these problems, we assume that the statistics of the motion field and the errors are not only spatially varying, but also unknown. We, thus, derive a blind adaptive technique based on generalized cross validation for estimating an independent regularization parameter for each pixel. Our formulation is pixelwise and combines existing first- and second-order constraints with a new second-order temporal constraint. We derive a new confidence measure for an adaptive rejection of erroneous and outlying motion vectors, and com- pare our results to other techniques in the literature. A new performance measure is also derived for estimating the signal-to-noise ratio for real sequences when the ground truth is unknown. [ABSTRACT FROM AUTHOR]