Search

360-degree/omnidirectional images (OIs) have received remarkable attention due to the increasing applications of virtual reality (VR). Compared to conventional 2D images, OIs can provide more immersive experiences to consumers, benefiting from the higher resolution and plentiful field of views (FoVs). Moreover, observing OIs is usually in a head-mounted display (HMD) without references. Therefore, an efficient blind quality assessment method, which is specifically designed for 360-degree images, is urgently desired. In this paper, motivated by the characteristics of the human visual system (HVS) and the viewing process of VR visual content, we propose a novel and effective no-reference omnidirectional image quality assessment (NR OIQA) algorithm by MultiFrequency Information and Local-Global Naturalness (MFILGN). Specifically, inspired by the frequency-dependent property of the visual cortex, we first decompose the projected equirectangular projection (ERP) maps into wavelet subbands by using discrete Haar wavelet transform (DHWT). Then, the entropy intensities of low-frequency and high-frequency subbands are exploited to measure the multifrequency information of OIs. In addition to considering the global naturalness of ERP maps, owing to the browsed FoVs, we extract the natural scene statistics (NSS) features from each viewport image as the measure of local naturalness. With the proposed multifrequency information measurement and local-global naturalness measurement, we utilize support vector regression (SVR) as the final image quality regressor to train the quality evaluation model from visual quality-related features to human ratings. To our knowledge, the proposed model is the first no-reference quality assessment method for 360-degree images that combines multifrequency information and image naturalness. Experimental results on two publicly available OIQA databases demonstrate that our proposed MFILGN outperforms state-of-the-art full-reference (FR) and NR approaches. [ABSTRACT FROM AUTHOR]

With the development of video surveillance in public safety field, there is an increasing research on person re-identification (re-id). In this paper, we address the image-to-video person re-id, in which the probe is an image and the gallery is consists of videos captured by nonoverlapping cameras. Compared with image, video sequence contains more temporal information that can be explored to improve the performance of re-identification system. However, it is challenging to model temporal information in the matching process of image-to-video person re-id. In this paper, we proposed a novel temporally memorized similarity learning neural network for this problem. In specific, the proposed network mainly consisted of two parts, including feature representation sub-network and similarity sub-network. In the first part, we adopted a convolutional neural network (CNN) to extract features from the input image. Given a video sequence of a person, features were first extracted from each its frame by using CNN and further forward to a long shot term memory (LSTM) network to encode the temporal information of video sequence. The outputs of LSTM were concatenated together as the feature vector of video sequences. Finally, the feature vectors of probe image and the video sequence were further forward to the similarity sub-network for distance metric learning. In the proposed framework, the feature representation and the similarity metric learning can be learned and optimized simultaneously. We evaluated the proposed framework on three public person re-id data sets, and the experimental results showed that the proposed approach is effective for the image-to-video person re-id. [ABSTRACT FROM AUTHOR]

Image segmentation is a key component of image analysis, which refers to the process of partitioning the image into multiple segments. Graph cut is widely used in image segmentation by constructing a graph that the minimal cut of this graph would lead to partition the corresponding pixels of the different objects. In this paper, we reconstruct the graph cut problem as a special non-convex optimization problem instead of the traditional maximum flow problem. We extend this non-convex problem to the hypergraph method and combine it with a tensor field based on a directional bilateral filter bank to achieve segmentation in grayscale images. Accordingly, an efficient minimization algorithm is proposed to solve this non-convex problem with global convergence. Furthermore, we have selected the data of BSDS300 and BSDS500 as tests. Experimental results and evaluation index tests further demonstrate the superiority of the proposed method. [ABSTRACT FROM AUTHOR]

Automated scene analysis has been a topic of great interest in computer vision and cognitive science. Recently, with the growth of crowd phenomena in the real world, crowded scene analysis has attracted much attention. However, the visual occlusions and ambiguities in crowded scenes, as well as the complex behaviors and scene semantics, make the analysis a challenging task. In the past few years, an increasing number of works on the crowded scene analysis have been reported, which covered different aspects including crowd motion pattern learning, crowd behavior and activity analyses, and anomaly detection in crowds. This paper surveys the state-of-the-art techniques on this topic. We first provide the background knowledge and the available features related to crowded scenes. Then, existing models, popular algorithms, evaluation protocols, and system performance are provided corresponding to different aspects of the crowded scene analysis. We also outline the available datasets for performance evaluation. Finally, some research problems and promising future directions are presented with discussions. [ABSTRACT FROM PUBLISHER]

In order to efficiently utilize the information in the data and eliminate the negative effects of outliers in the principal component analysis (PCA) method, in this paper, we propose a novel robust sparse PCA method based on maximum correntropy criterion (MCC) with high-order manifold constraints called the RHSPCA. Compared with the traditional PCA methods, the proposed RHSPCA has the following benefits: 1) the MCC regression term is more robust to outliers than the MSE-based regression term; 2) thanks to the high-order manifold constraints, the low-dimensional representations can preserve the local relations of the data and greatly improve the clustering and classification performance for image processing tasks; and 3) in order to further counteract the adverse effects of outliers, the MCC-based samples’ mean is proposed to better centralize the data. We also propose a new solver based on the half-quadratic technique and accelerated block coordinate update strategy to solve the RHSPCA model. Extensive experimental results show that the proposed method can outperform the state-of-the-art robust PCA methods on a variety of image processing tasks, including reconstruction, clustering, and classification, on outliers contaminated datasets. [ABSTRACT FROM AUTHOR]

Objectness has recently become a standard step in many computer vision tasks. Among various techniques, those based on hierarchical image segmentation play a fundamental role for developments in new data modalities. In this paper, we address the problem of objectness in RGB-D images and propose a novel and effective approach, namely, hierarchical image segmentation ensemble (HISE). Different from existing image segmentation based methods that generate object segments or proposals largely by heuristics or empirical rules, HISE learns superpixel mergings with a hierarchical tree-structured ensemble, where individual merging models of the ensemble are formed by traversing different paths of the tree, and where both the merging accuracy and proposal diversity are emphasized. Furthermore, we use efficient feature measurements that support easy integration of additional clues. Extensive experiments conducted on the benchmark NYU-v2 RGB-D and SUN RGB-D data sets show the competency of our proposed method. [ABSTRACT FROM AUTHOR]

In this paper, two metrics for measuring image sharpness are presented and used for an autofocus (AF) application. Both measures exploit reorganized discrete cosine transform (DCT) representation. The first metric is a focus measure, which involves optimal high- and middle-frequency coefficients to evaluate relative sharpness. It is robust to noise while remaining sensitive to the best focus position. A psychometric function-based metric is introduced to quantify the focus measure. The second metric is a no-reference blurriness metric, which is used to measure absolute blurriness. It first constructs multiscale DCT edge maps using directional energy information and then determines image blurriness by combining change information in edge structures with image contrast. This metric gives predictions that are closely correlated with subjective perceived scores and shows performance comparable with that of state-of-the-art methods, especially for noisy images. For noisy situations, the two metrics are adjusted adaptively according to the estimated noise level. To prevent the introduction of extra computational load, an efficient noise-level estimation algorithm based on median absolute deviation is presented. This algorithm exploits only the available reorganized DCT coefficients. With the focus and blurriness measures, an AF method for which the two metrics play an important role was developed. Because of their high-quality performance, the realized AF function is able to locate the best focus position swiftly and reliably. [ABSTRACT FROM PUBLISHER]

Visual tracking in image streams acquired by static cameras is usually based on change detection and recursive Bayesian estimation, such an approach laying at the core of many practical applications. Yet, the interaction between the change detector and the Bayesian filter is typically designed heuristically. Differently, this paper develops a sound framework to model and implement a bidirectional communication flow between the two processes. In our Bayesian loop, change detection provides well-defined observation likelihood to the recursive filter and the filter prediction provides an informative prior to the change detector, which deploys Bayesian reasoning alike. The loop is developed for the two major variants of Bayesian filters used in tracking, namely the Kalman filter and the particle filter. Experiments on publicly available videos and a novel challenging data set show that the proposed interaction scheme outperforms several state-of-the-art trackers. [ABSTRACT FROM AUTHOR]

This paper is dedicated to the development of a thermal monitoring system for microelectronics based on the attentive vision approach as applied to image sequence analysis using an infrared camera. The attentive vision method implements multiscale image sequence analysis by a spatiotemporal attention operator to detect feature points, which are located inside potential thermal stress regions. The attention operator is a linear aggregation of temporal change and spatial saliency filters. The monitoring process is organized in two hierarchical phases: 1) peripheral and 2) focal. The focal monitoring is mostly carried out through the tracking of stress-relevant feature-point areas and analysis of their spatiotemporal descriptors. The thermal monitoring experiments conducted with wafer-scale integrated circuits have confirmed the reliability of the proposed approach and showed its high potential in image sequence analysis for monitoring purposes. [ABSTRACT FROM PUBLISHER]

Vehicle detection in aerial images is a crucial image processing step for many applications such as screening of large areas as used for surveillance, reconnaissance, or rescue tasks. In recent years, several deep learning-based frameworks have been proposed for object detection. However, these detectors were developed for data sets that considerably differ from aerial images. In this paper, we systematically investigate the potential of fast R-CNN and faster R-CNN for aerial images, which achieve top performing results on common detection benchmark data sets. Therefore, the applicability of eight state-of-the-art object proposal methods used to generate a set of candidate regions and of both detectors is examined. Relevant adaptations to account for the characteristics of the aerial images are provided. To overcome the shortcomings of the original approach in the case of handling small instances, we further propose our own networks that clearly outperform state-of-the-art methods for vehicle detection in aerial images. Furthermore, we analyze the impact of the different adaptations with respect to various ground sampling distances to provide a guideline for detecting small objects in aerial images. All experiments are performed on two publicly available data sets to account for differing characteristics such as varying object sizes, number of objects per image, and varying backgrounds. [ABSTRACT FROM AUTHOR]

In this paper, a 2-D noncausal Markov model is proposed for passive digital image-splicing detection. Different from the traditional Markov model, the proposed approach models an image as a 2-D noncausal signal and captures the underlying dependencies between the current node and its neighbors. The model parameters are treated as the discriminative features to differentiate the spliced images from the natural ones. We apply the model in the block discrete cosine transformation domain and the discrete Meyer wavelet transform domain, and the cross-domain features are treated as the final discriminative features for classification. The support vector machine which is the most popular classifier used in the image-splicing detection is exploited in our paper for classification. To evaluate the performance of the proposed method, all the experiments are conducted on public image-splicing detection evaluation data sets, and the experimental results have shown that the proposed approach outperforms some state-of-the-art methods. [ABSTRACT FROM PUBLISHER]

In stereo matching, it is a challengeable task to effectively deal with occlusions between objects, which often cause disparity errors along object boundaries. In this paper, we propose discontinuity-preserving stereo matching based on reliability. We first estimate an initial disparity map by segmentation-based adaptive weights. Then, we obtain reliability for each pixel in an image, which indicates the correctness of the estimated disparity based on the matching cost. Finally, we minimize disparity errors caused by occlusions between objects using reliability and produce discontinuity-preserving disparity maps. Experiments on the Middlebury data set show that the proposed method successfully preserves disparity discontinuities along object boundaries and outperforms the state-of-the-art stereo matching methods. [ABSTRACT FROM AUTHOR]

This paper proposes novel noniterative mean-shift-based image segmentation that uses global and local attributes. The existing mean-shift-based methods use a fixed range bandwidth, and hence their accuracy is dependent on the range spectrum of an image. To resolve this dependency, this paper proposes to modify the range kernel in the mean-shift process to be anisotropic. The modification is conducted using a global attribute defined as the range covariance matrix of the image. Further, to alleviate oversegmentation, the proposed method merges the segments having similar local attributes more aggressively than other segments. The local attribute for each segment is defined as the sum of the variances of the chromatic components. Finally, to expedite the processing, the proposed method uses a region adjacency graph (RAG) for the merging process, thus differing from the existing linked mean-shift-based methods. In the experiments on the Berkeley segmentation data set, the use of the global and local attributes improved segmentation accuracy; the proposed method outperformed the state-of-the-art linked mean-shift-based method by showing an improvement of 2.15%, 3.16%, 3.32%, and 1.90% in probability rand index, segmentation covering, variation of information, and F-measure, respectively. Further, compared with the benchmark method, which uses the dilating and merging scheme, the proposed method improved the speed of the merging process 42 times by applying the RAG. [ABSTRACT FROM PUBLISHER]

In this paper, we propose a novel method for 4D light-field (LF) depth estimation exploiting the special linear structure of an epipolar plane image (EPI) and locally linear embedding (LLE). Without high computational complexity, depth maps are locally estimated by locating the optimal slope of each line segmentation on the EPIs, which are projected by the corresponding scene points. For each pixel to be processed, we build and then minimize the matching cost that aggregates the intensity pixel value, gradient pixel value, spatial consistency, as well as reliability measure to select the optimal slope from a predefined set of directions. Next, a subangle estimation method is proposed to further refine the obtained optimal slope of each pixel. Furthermore, based on a local reliability measure, all the pixels are classified into reliable and unreliable pixels. For the unreliable pixels, LLE is employed to propagate the missing pixels by the reliable pixels based on the assumption of manifold preserving property maintained by natural images. We demonstrate the effectiveness of our approach on a number of synthetic LF examples and real-world LF data sets, and show that our experimental results can achieve higher performance than the typical and recent state-of-the-art LF stereo matching methods. [ABSTRACT FROM PUBLISHER]

A local line-like feature is the most important discriminate information in the image upsampling scenario. In recent example-based upsampling methods, grayscale and gradient features are often adopted to describe the local patches, but these simple features cannot accurately characterize complex patches. In this paper, we present a feature representation of local edges by means of a multilevel filtering network, namely, multilevel modified finite Radon transform network (MMFRTN). In the proposed MMFRTN, the MFRT is utilized in the filtering layer to extract the local line-like feature; the nonlinear layer is set to be a simple local binary process; for the feature-pooling layer, we concatenate the mapped patches as the feature of local patch. Then, we propose a new example-based upsampling method by means of the MMFRTN feature. Experimental results demonstrate the effectiveness of the proposed method over some state-of-the-art methods. [ABSTRACT FROM AUTHOR]

In this paper, we address the segmentation problem under limited computation and memory resources. Given a segmentation algorithm, we propose a framework that can reduce its computation time and memory requirement simultaneously, while preserving its accuracy. The proposed framework uses standard pixel-domain downsampling and includes two main steps. Coarse segmentation is first performed on the downsampled image. Refinement is then applied to the coarse segmentation results. We make two novel contributions to enable competitive accuracy using this simple framework. First, we rigorously examine the effect of downsampling on segmentation using a signal processing analysis. The analysis helps to determine the uncertain regions, which are small image regions where pixel labels are uncertain after the coarse segmentation. Second, we propose an efficient minimum spanning tree-based algorithm to propagate the labels into the uncertain regions. We perform extensive experiments using several standard data sets. The experimental results show that our segmentation accuracy is comparable to state-of-the-art methods, while requiring much less computation time and memory than those methods. [ABSTRACT FROM PUBLISHER]

High-quality magnetic resonance (MR) images afford more detailed information for reliable diagnoses and quantitative image analyses. Given low-resolution (LR) images, the deep convolutional neural network (CNN) has shown its promising ability for image super-resolution (SR). The LR MR images usually share some visual characteristics: structural textures of different sizes, edges with high correlation, and less informative background. However, multi-scale structural features are informative for image reconstruction, while the background is more smooth. Most previous CNN-based SR methods use a single receptive field and equally treat the spatial pixels (including the background). It neglects to sense the entire space and get diversified features from the input, which is critical for high-quality MR image SR. We propose a wide weighted attention multi-scale network ($\text{W}^{2}$ AMSN) for accurate MR image SR to address these problems. On the one hand, the features of varying sizes can be extracted by the wide multi-scale branches. On the other hand, we design a non-reduction attention mechanism to recalibrate feature responses adaptively. Such attention preserves continuous cross-channel interaction and focuses on more informative regions. Meanwhile, the learnable weighted factors fuse extracted features selectively. The encapsulated wide weighted attention multi-scale block ($\text{W}^{2}$ AMSB) is integrated through a recurrent framework and global attention mechanism. Extensive experiments and diversified ablation studies show the effectiveness of our proposed $\text{W}^{2}$ AMSN, which surpasses state-of-the-art methods on most popular MR image SR benchmarks quantitatively and qualitatively. And our method still offers superior accuracy and adaptability on real MR images. [ABSTRACT FROM AUTHOR]

Every year, a large number of wildfires all over the world burn forested lands, causing adverse ecological, economic, and social impacts. Beyond taking precautionary measures, early warning and immediate response are the only ways to avoid great losses. To this end, in this paper we propose a computer vision approach for fire-flame detection to be used by an early-warning fire monitoring system. Initially, candidate fire regions in a frame are defined using background subtraction and color analysis based on a nonparametric model. Subsequently, the fire behavior is modeled by employing various spatio-temporal features, such as color probability, flickering, spatial, and spatio-temporal energy, while dynamic texture analysis is applied in each candidate region using linear dynamical systems and a bag-of-systems approach. To increase the robustness of the algorithm, the spatio-temporal consistency energy of each candidate fire region is estimated by exploiting prior knowledge about the possible existence of fire in neighboring blocks from the current and previous video frames. As a final step, a two-class support vector machine classifier is used to classify the candidate regions. Experimental results have shown that the proposed method outperforms existing state-of-the-art algorithms. [ABSTRACT FROM PUBLISHER]

Homography estimation is a fundamental problem in the field of computer vision. For estimating the homography between two images, one of the key issues is to match keypoints in the reference image to the keypoints in the moving image. To match keypoints in real time, a binary image descriptor, due to its low matching and storage costs, emerges as a more and more popular tool. Upon achieving the low costs, the binary descriptor sacrifices the discriminative power of using floating points. In this paper, we present BB-Homography, a new approach that fuses fast binary descriptor matching and bipartite graph for homography estimation. Starting with binary descriptor matching, BB-Homography uses bipartite graph matching (GM) algorithm to refine the matching results, which are finally passed over to estimate homography. On realizing the correlation between keypoint correspondence and homography estimation, BB-Homography iteratively performs the GM and the homography estimation such that they can refine each other at each iteration. In particular, based on spectral graph, a fast bipartite GM algorithm is developed for lowering the time cost of BB-Homography. BB-Homography is extensively evaluated on both public benchmarks and live-captured video streams that consistently shows that BB-Homography outperforms conventional methods for homography estimation. [ABSTRACT FROM PUBLISHER]

The popularity of mobile applications has greatly enriched and facilitated our lives. However, the rapid increase of digital images and the problem of narrow bandwidth of the wireless network call for an appropriate approach to reduce the amount of data transmitted over the wireless network (i.e., low bit-rate transmission) while ensuring high recognition accuracy at the cloud. We propose a simple and effective feature retargeting (FR) approach for retargeting an image while preserving the representative local features (e.g., SIFT, SURF, and BRIEF) in the image. Our feature retargeting approach aims at low bit-rate visual recognition instead of high-quality visual perception that visual retargeting methods dedicate to. Our algorithm consists of two key novelties: estimating feature saliency and retargeting image: Estimating feature saliency focuses on predicting the relative importance of different features in an image by analyzing uniqueness in a specific context; Retargeting image aims at finding the optimal resolution for the retargeted image to maximize feature-saliency energy. We evaluate the proposed approach for two different applications in three large data sets and observe that our FR approach consistently outperforms state-of-the-art retargeting algorithms, resulting in both higher precision and lower bit-rates. We also demonstrate that even when the resolution of source image is reduced greatly, e.g., 1/7 original size, our algorithm produces superior results as compared with other approaches. [ABSTRACT FROM AUTHOR]

The autoregressive (AR) model has been widely used in signal processing for its effective estimation, especially in image processing. Many dedicated $2\times $ interpolation algorithms adopt the AR model to describe the strong correlation between low-resolution (LR) pixels and high-resolution (HR) pixels. However, these AR model-based methods closely depend on the fixed relative position between LR pixels and HR pixels that are nonexistent in the general scale interpolation. In this paper, we present an adaptive general scale interpolation algorithm that is capable of arbitrary scaling factors considering the nonstationarity of natural images. Different from other dedicated $2\times $ interpolation methods, the proposed AR terms are modeled by pixels with their adjacent unknown HR neighbors. To compensate for the information loss caused by mismatches of AR models, we consider a weighting scheme suitable for general scale situations based on the pixel similarity to increase accuracy of the estimation. Comprehensive experiments demonstrate the effectiveness of the proposed method on general scaling factors. The maximum gain of peak signal-to-noise ratio is 2.07 dB compared with segment adaptive gradient angle in $1.5 \times $ enlargements. To evaluate the performance in resolution adaptive video coding, we have also tested our method on Joint Scalable Video Model codec and obtained better subjective quality and rate-distortion performance. [ABSTRACT FROM PUBLISHER]

This paper investigates a general framework to discover categories of unlabeled scene images according to their appearances (i.e., textures and structures). We jointly solve the two coupled tasks in an unsupervised manner: 1) classifying images without predetermining the number of categories and 2) pursuing generative model for each category. In our method, each image is represented by two types of image descriptors that are effective to capture image appearances from different aspects. By treating each image as a graph vertex, we build up a graph and pose the image categorization as a graph partition process. Specifically, a partitioned subgraph can be regarded as a category of scenes and we define the probabilistic model of graph partition by accumulating the generative models of all separated categories. For efficient inference with the graph, we employ a stochastic cluster sampling algorithm, which is designed based on the Metropolis–Hasting mechanism. During the iterations of inference, the model of each category is analytically updated by a generative learning algorithm. In the experiments, our approach is validated on several challenging databases, and it outperforms other popular state-of-the-art methods. The implementation details and empirical analysis are presented as well. [ABSTRACT FROM PUBLISHER]

Intrinsic image analysis plays an important role for image understanding, since it can provide accurate reflectance, shape and illumination information of the scene. However, intrinsic image analysis is an ill-posed problem which need to apply extra constrains for the decomposition of reflectance image and shading image from a single image. Recently deep neural networks are introduced for intrinsic image analysis, which can produce two intrinsic components simultaneously. In fact, the mutually exclusive relationship between reflectance image and shading image is not only a constraint for decomposition but also can improve the decomposition results. However, this relationship is always omitted in the current networks. In order to address this problem, we propose a novel deep network called as Cascaded Quotient Network (CasQNet) for intrinsic image decomposition. The CasQNet consists of two sub-networks: a Pyramid Mini-U-Net (PyNet) that specifically extracts the reflectance image in multi-scale and a Shading Optimization Network (SoNet) that optimizes the resulting shading. These two sub-networks are cascaded by a quotient operation, which directly enforces the mutually exclusive relationship between reflectance image and shading image in the network architecture. In PyNet, the task of reconstructing reflectance image is achieved by a series of nested multi-scale U-Nets, which simplified the learning task for each U-Net. SoNet is designed to address the unsmooth and blur problems of extreme points caused by the quotient operation. PyNet and SoNet are trained alternately and finally jointed in cascaded structure. Furthermore, we combine multiple loss functions, which consist of data loss, correlation loss and reconstruction loss, for improving the learning effectiveness. To evaluate our proposed algorithm, extensive experiments are performed on three datasets, i.e., ShapeNet, BOLD Surface and MIT Intrinsic Image datasets. Qualitative and quantitative results show that our model achieves the best performance compared to the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

This paper proposes a new approach for mobile landmark recognition based on integrated content and context analysis. Conventional scene/landmark recognition methods focus mainly on nonmobile desktop/PC platform, where content analysis alone is used to perform landmark recognition. These nonmobile systems, however, do not take unique features of mobile devices into consideration, e.g., limited computational power and fast response time requirement of mobile users. On the contrary, most existing context-aware content mobile landmark recognition methods mainly rely on global positioning system location information for context analysis. In view of this, this paper proposes an effective method that employs an integration of content and context analysis to perform landmark recognition using mobile devices. A new bags-of-words (BoW) framework is developed to perform content analysis. It is then integrated with context analysis involving fusion of location and direction information to perform mobile landmark recognition. Experimental results based on the NTU50Landmark database show that the proposed method can achieve good recognition performance in mobile landmark recognition. [ABSTRACT FROM AUTHOR]

A (k,n) visual cryptographic scheme (VCS) encodes a secret image into n shadow images (printed on transparencies) distributed among n participants. When any k participants superimpose their transparencies on an overhead projector (OR operation), the secret image can be visually revealed by a human visual system without computation. However, the monotone property of OR operation degrades the visual quality of reconstructed image for OR-based VCS (OVCS). Accordingly, XOR-based VCS (XVCS), which uses XOR operation for decoding, was proposed to enhance the contrast. In this paper, we investigate the relation between OVCS and XVCS. Our main contribution is to theoretically prove that the basis matrices of (k,n)-OVCS can be used in (k,n)-XVCS. Meantime, the contrast is enhanced 2^(k-1) times. [ABSTRACT FROM PUBLISHER]

Image forensics has attracted wide attention during the past decade. However, most existing works aim at detecting a certain operation, which means that their proposed features usually depend on the investigated image operation and they consider only binary classification. This usually leads to misleading results if irrelevant features and/or classifiers are used. For instance, a JPEG decompressed image would be classified as an original or median filtered image if it was fed into a median filtering detector. Hence, it is important to develop forensic methods and universal features that can simultaneously identify multiple image operations. Based on extensive experiments and analysis, we find that any image operation, including existing anti-forensics operations, will inevitably modify a large number of pixel values in the original images. Thus, some common inherent statistics such as the correlations among adjacent pixels cannot be preserved well. To detect such modifications, we try to analyze the properties of local pixels within the image in the residual domain rather than the spatial domain considering the complexity of the image contents. Inspired by image steganalytic methods, we propose a very compact universal feature set and then design a multiclass classification scheme for identifying many common image operations. In our experiments, we tested the proposed features as well as several existing features on 11 typical image processing operations and four kinds of anti-forensic methods. The experimental results show that the proposed strategy significantly outperforms the existing forensic methods in terms of both effectiveness and universality. [ABSTRACT FROM PUBLISHER]

Demosaicking and denoising are key steps in the camera imaging chain for both images and videos. The reconstruction errors during these stages will have undesirable effects on the final result if not handled properly. Demosaicking provokes the spatial and color correlation of noise, which is afterwards enhanced by the processing pipeline. This structured noise generally degrades the image quality and, for dark scenes with low signal to noise ratio, prevents the correct interpretation of the image. When trying to mitigate such structured noise on already processed data, denoising methods attenuate details and texture. We present a video processing chain, consisting of a novel strategy for the removal of noise at the camera sensor and a novel video demosaicking algorithm. In both cases, a spatio-temporal patch-based filter with motion compensation is introduced. The experimental results, including real examples, illustrate the performance of the proposed chain, avoiding the creation of interpolation artefacts and colored spots. [ABSTRACT FROM AUTHOR]

This paper presents a novel multiview gait recognition method that combines the enhanced Gabor (EG) representation of the gait energy image and the regularized local tensor discriminant analysis (RLTDA) method. EG first derives desirable gait features characterized by spatial frequency, spatial locality, and orientation selectivity to cope with the variations due to surface, shoe types, clothing, carrying conditions, and so on. Unlike traditional Gabor transformation, which does not consider the structural characteristics of the gait features, our representation method not only considers the statistical property of the input features but also adopts a nonlinear mapping to emphasize those important feature points. The dimensionality of the derivation of EG gait feature is further reduced by using RLTDA, which directly obtains a set of locally optimal tensor eigenvectors and can capture nonlinear manifolds of gait features that exhibit appearance changes due to variable viewing angles. An aggregation scheme is adopted to combine the complementary information from differently RLTDA recognizers at the matching score level. The proposed method achieves the best average Rank-1 recognition rates for multiview gait recognition based on image sequences from the USF HumanID gait challenge database and the CASIA gait database. [ABSTRACT FROM AUTHOR]

This paper proposes a new framework for joint image registration and high-resolution (HR) image reconstruction from multiple low-resolution (LR) observations with zooming motion. Conventional super-resolution (SR) methods typically formulate the SR problem as a two-stage process, namely, image registration followed by HR reconstruction. An important step in image SR is the effective estimation of motion parameters. However, the registration algorithms in these two-stage processes experience various degrees of errors. This could degrade the quality of subsequent HR reconstruction. In view of this, this paper presents a new approach that performs joint image registration and SR reconstruction. The proposed iterative SR framework enables the HR image and motion parameters to be estimated simultaneously and progressively. This could increase the potential SR improvement as more accurate estimates of motion parameters could be obtained iteratively. Experimental results show that the proposed method is effective in performing image registration and SR for simulated and real-life images and videos. [ABSTRACT FROM AUTHOR]

In this paper, a new quaternion vector filter for removal of random impulse noise in color video sequences is presented. First, luminance distances and chromaticity differences that are represented in quaternion form are combined together to measure color distances between color pixels. Then, based on this new color distance mechanism, the samples along horizontal, vertical, and diagonal directions in current frame and the samples of adjacent frames on motion trajectory are used to detect whether each pixel is noisy or not. By analyzing the spatiotemporal order-statistic information about these directional samples, the video pixels are classified into noise free and noisy. Finally, 3-D weighted vector median filtering is performed on the pixels that are judged as noisy, and the other pixels remain unchanged. The experimental results show that the proposed algorithm significantly outperforms other state-of-the-art video denoising methods in terms of both objective measure and visual evaluation. [ABSTRACT FROM PUBLISHER]

This paper presents a novel algorithm for the removal of reflections generated by objects on reflecting floors. The algorithm uses both chromatic properties of the reflections and geometrical constraints on their positions; however, it does not make use of a model of the reflected objects, and so it can be applied to scenes containing several kinds of objects (e.g., people, baggage, animals, vehicles, etc.). The proposed method has been validated by an extensive set of experiments on a large video database. In these experiments, the method has been compared to two other recent reflection removal algorithms. The experimental results show that the proposed method is fast and effective, both in absolute terms and in comparison with the other algorithms. [ABSTRACT FROM AUTHOR]

One of the remaining problems in iris recognition is the implementation of its efficient hardware systems. The difficulty arises from the fact that the methods forming part of this type of authentication process are fairly complex. Furthermore, next generation algorithms for iris recognition will become even more complex to enhance the reliability and functionality of currently used solutions. An example of such an intensive processing task is iris image quality assessment and segmentation using video signal obtained at-a-distance and on-the-move. Thus, processing robustness and predictable analysis time in such dedicated architectures may be particularly important. This paper describes a hardware system designed to analyze iris biometric data developed as a part of the work conducted by the authors on the complete iris identification system (1:N). This specialized architecture is mainly composed of digital signal processors and field-programmable gate arrays. Several issues concerning efficient biometric sample processing are discussed. Algorithms for iris recognition previously developed by the authors on a PC platform were adapted to the described architecture. The obtained results are presented, and the developed system is compared with chosen commercially available iris authentication systems. [ABSTRACT FROM AUTHOR]

This paper proposes a novel kernel similarity modeling of texture pattern flow (KSM-TPF) for background modeling and motion detection in complex and dynamic environments. The texture pattern flow encodes the binary pattern changes in both spatial and temporal neighborhoods. The integral histogram of texture pattern flow is employed to extract the discriminative features from the input videos. Different from existing uniform threshold based motion detection approaches which are only effective for simple background, the kernel similarity modeling is proposed to produce an adaptive threshold for complex background. The adaptive threshold is computed from the mean and variance of an extended Gaussian mixture model. The proposed KSM-TPF approach incorporates machine learning method with feature extraction method in a homogenous way. Experimental results on the publicly available video sequences demonstrate that the proposed approach provides an effective and efficient way for background modeling and motion detection. [ABSTRACT FROM PUBLISHER]

This paper presents a novel, real-time digital image stabilization (DIS) system using a modified proportional integrated (MPI) controller to remove stably unwanted shaking from an image sequence that is captured by a hand-held video camera without affecting the deliberate panning motion of the camera. Researchers have addressed the trade-off problem between the removal of shaking and the preservation of global camera movement. This paper analyzes motion compensation, which is formulated as a control problem. A compensating motion vector (CMV) estimation method with a modified PI control system is proposed both to remove the unwanted jitter and to preserve the deliberate, panning motion of the camera. Experimental results demonstrate that the proposed system provides robust motion compensation of image sequences in various conditions. [ABSTRACT FROM AUTHOR]

Image inpainting or image completion is the technique that automatically restores/completes removed areas in an image. When dealing with a similar problem in video, not only should a robust tracking algorithm be used, but the temporal continuity among video frames also needs to be taken into account, especially when the video has camera motions such as zooming and tilting. In this paper, we extend an exemplar-based image inpainting algorithm by incorporating an improved patch matching strategy for video inpainting. In our proposed algorithm, different motion segments with different temporal continuity call for different candidate patches, which are used to inpaint holes after a selected video object is tracked and removed. The proposed new video inpainting algorithm produces very few "ghost shadows," which were produced by most image inpainting algorithms directly applied on video. Our experiments use different types of videos, including cartoon, video from games, and video from digital camera with different camera motions. Our demonstration at http://member.mine.tku.edu.tw/www/T_CSVT/web/shows the promising results. [ABSTRACT FROM AUTHOR]

During the last years, the task of automatic event analysis in video sequences has gained an increasing attention among the research community. The application domains are disparate, ranging from video surveillance to automatic video annotation for sport videos or TV shots. Whatever the application field, most of the works in event analysis are based on two main approaches: the former based on explicit event recognition, focused on finding high-level, semantic interpretations of video sequences, and the latter based on anomaly detection. This paper deals with the second approach, where the final goal is not the explicit labeling of recognized events, but the detection of anomalous events differing from typical patterns. In particular, the proposed work addresses anomaly detection by means of trajectory analysis, an approach with several application fields, most notably video surveillance and traffic monitoring. The proposed approach is based on single-class support vector machine (SVM) clustering, where the novelty detection SVM capabilities are used for the identification of anomalous trajectories. Particular attention is given to trajectory classification in absence of a priori information on the distribution of outliers. Experimental results prove the validity of the proposed approach. [ABSTRACT FROM AUTHOR]

This paper addresses the problem of deriving measures that express the degree of the reliability of motion vectors estimated by a block-matching motion estimation method. We express the block matching motion estimation scheme in the probabilistic framework as a maximum-likelihood estimation scheme. Subsequently, we derive the confidence measures in terms of the a posteriori probabilities and the likelihoods of the estimated vectors. The assumptions about the type of the likelihood, that is, about the underlying conditional probability distribution of the motion compensated intensity differences (e.g., Laplacian) are derived from the objective criterion of the block-matching estimator. All parameters are estimated from data that are derived as by-product of the motion estimation scheme and our method, practically, introduces no additional computational cost. The derivation of the confidence measures is incorporated in a multiscale scheme. Experimental results are presented for image sequences with known ground-truth motion. [ABSTRACT FROM AUTHOR]

Prominences are clouds of relatively cool and dense gas in the solar atmosphere. In this paper, we present a new method to detect and characterize the prominence eruptions. The input is a sequence of consecutive Hα solar images, and the output is a list of prominence eruption events detected. We extract the limb events and measure their associated properties by applying image processing techniques. First, we perform image normalization and noise removal. Then, we isolate the limb objects and identify the prominence features. Finally, we apply pattern recognition techniques to classify the eruptive prominences. The characteristics of prominence eruptions, such as brightness, angular width, radial height and velocity are measured. The method presented can lead to automatic monitoring and characterization of solar events. [ABSTRACT FROM AUTHOR]

In this paper, a simple and effective fast block matching algorithm (BMA) is proposed, called adaptive irregular pattern search with matching prejudgment (ATPS-MP). In the ALPS-MP, a dynamic search pattern, adaptive irregular pattern (AlP), is constructed for each block based on its spatial and temporal neighboring motion vectors (MVs). The AlP is used to quickly identify the best center point for the follow-up refined search to effectively reduce unnecessary intermediate searches and avoid the search trapping at local-minimum matching error position. The construction of the ALP jointly takes the tradeoff of rate and distortion into consideration and favors the differential coding of MVs. The matching prejudgment (MP) is iinbedded in the AlPS and exploits an adaptive threshold determined for each block based on the prediction error of the temporal neighboring block. The simulation results show that the proposed ALPS-MP consistently achieves very close, and sometimes even higher, peak signal-to-noise ratio (PSNR) than the full search, while achieving 167.6-871.5 times of speed-up or computational gain, measured in terms of the average number of search points per MV generation. It also greatly outperforms two well-referenced fast BMAs, the diamond search and the motion vectorfield adaptive search technique (MVFAST), adopted in the MPEG-4 Verification Model and the Optimization Model, respectively, on the average PSNR and the search speed. [ABSTRACT FROM AUTHOR]

It is well known that low bit rate block-based discrete cosine transform coded image exhibits visually annoying coding artifacts. In this paper, we proposed a projection onto convex sets (PCOS)-based deblocking algorithm using a novel region smoothness constraint set for graphic images containing objects with smooth regions. The smoothness constraint set is obtained by an explicit modeling of smooth regions in the image using a spatially adaptive thin-plate spline. In contrast to most deblocking algorithms which enforce smoothness just around the 8 x 8 block boundaries, our algorithm enforces smoothness in regions which could possibly span several blocks. We showed that convergence of our algorithm could be reached within one iteration. The performance of the proposed algorithm is evaluated visually and quantitatively in term of peak signal-to-noise ratios and the mean squared difference of slope metric, which measures the impact of the blocking effects, for several graphic images. The results show that our algorithm can effectively suppress blockiness in smooth regions while still preserving the sharpness of object edges. [ABSTRACT FROM AUTHOR]

In video encoding, block motion estimation represents a CPU-intensive task. For this reason, many fast algorithms have been developed to Improve searching and matching phases. A mile- stone within the lossless approach is partial distortion elimination (PDE/SpiraIPDE) In which distortion is the difference between the block to be coded and the candidate prediction block. In this paper, (i) we analyze distortion behavior from. loéal information using the Thylor series expansion md show that our general analysis includes other previous similar approaches. (ii) Then, we propose two full-search (lossless), fast-matching, block motion estimation algorithms, based on the PDE idea. The proposed algorithms, called fast full search with sorting by distortion (FFSSU) Sd" fast full search with sorting by gradient (FFSSG), sort the contributions to distortion and the gradient values, respectively, in order to. quickly discard invalid blocks. Experimental results show that the pro- posed algorithms outperform other existing full search algorithms, reducing by up to 20% the total CPU encoding time (with respect to Spira1PDE), while the computation strictly required by the motion estimation is reduced by about 30%. (lII) Finally, we experimentally find an operational lower bound (based on standard test sequences) for the average number of checked pixels in the PDE approach, which measures the performance of the searching and matching phases. In particular, SpIraIPDE achieves performances very close to the searching phase bound, while there is still a remarkable margin on the matching phase. We then show that our algorithms, aimed at improving the performances of the matching phase, achieve interesting results, significantly approaching this margin. [ABSTRACT FROM AUTHOR]

Multiple description coding (MDC) is a source coding technique that exploits path diversity to increase the robustness of transmitting a compressed signal over error-prone channels. However, the application of MDC to video coding is still problematic because a prediction mismatch problem, called drift, may occur at the decoder when one description is lost. In the literature, it has been shown that drift can be effectively prevented by encoding this prediction mismatch and sending it as side information in a three- prediction-loop structure. However, this side information is totally redundant when both descriptions are received. In this paper, we propose a very simple, but efficient two-stage MDC in the central prediction loop based on Reibman's three prediction-loop structure. In our design, an additional multiple description scalar quantizer is introduced into the central prediction loop. The encoding of the side information is modified in such a way that it can be used to improve the video quality when no drift occurs, but still maintain drift prevention capability. The simulation results with different video sequences at various packet loss rates demonstrate that the proposed two-stage MDC structure indeed enhances the quality when the channel is error-free, yet effectively avoids drift when one description is completely lost. [ABSTRACT FROM AUTHOR]

We present a fully automatic system for face recognition in databases with only a small number of samples (even single sample) for each individual. In this paper, the shape localization problem is formulated in the Bayesian framework. In the learning stage, the RankBoost approach is introduced to model the likelihood of local features associated with the fiducial point, while preserving the prior ranking order between the ground truth position and its neighbors; in the inferring stage, a simple efficient iterative algorithm is proposed to uncover the MAP shape by locally modeling the likelihood distribution around each fiducial point. Based on the accurately located fiducial points, two popular mutual enhancing texture features are automatically extracted and integrated for human face representation: global texture features, which are the normalized shape-free gray-level values enclosed in the mean shape, and local texture features, which are represented by the Gabor wavelets extracted at the fiducial points (eye corners and mouth, etc.). Global texture mainly encodes the low-frequency information of a face, while local texture encodes the local high-frequency components. The extensive experiments illustrate that our proposed shape localization approach significantly improved the shape location accuracy, robustness, and face recognition rate; moreover, the experiments conducted on FERET and Yale databases show that our algorithm outperformed the classical eigenfaces, fisherfaces, as well as other approaches utilizing the shape, global, and local textures. [ABSTRACT FROM AUTHOR]

The creation of novel views using prestored images or image-based rendering has many potential applications, such as visual simulation, virtual reality, and telepresence, for which traditional computer graphics based on geometric modeling would be unsatisfactory particularly with very complex three-dimensional scenes. This paper presents a new image-based rendering system that tackles the two most difficult problems of image-based modeling: pixel matching and visibility handling. We first introduce the joint view triangulation (JVT), a novel representation for pairs of images that handles the visibility and occlusion problems created by the parallaxes between the images. The JVT is built from matched planar patches regularized by local smooth constraints encoded by plane homographies. Then, we introduce an incremental edge-constrained construction algorithm. Finally, we present a pseudo-painter's rendering algorithm for the JVT and demonstrate the performance of these methods experimentally. [ABSTRACT FROM AUTHOR]

Image-based rendering (IBR) has become a very active research area in recent years. The spectral analysis problem for IBR has not been completely solved. In this paper, we present a new method to parameterize the problem, which is applicable for general-purpose IBR spectral analysis. We notice that any plenoptic function is generated by light ray emitted/reflected/refracted from the object surface. We introduce the surface plenoptic function (SPF), which represents the light rays starting from the object surface. Given that radiance along a light ray does not change unless the light ray is blocked, SPF reduces the dimension of the original plenoptic function to 6D. We are then able to map or transform the SPF to IBR representations captured along any camera trajectory. Assuming some properties on the SPF, we can analyze the properties of IBR for generic scenes such as scenes with Lambertian or non-Lambertian surfaces and scenes with or without occlusions, and for different sampling strategies such as lightfieid/concentric mosaic. We find that in most cases, even though the SPF may be band-limited, the frequency spectrum of IBR is not band-limited. We show that non-Lambertian reflections, depth variations and occlusions can all broaden the spectrum, with the latter two being more significant.SPF is defined for scenes with known geometry. When the geometry is unknown, spectral analysis is still possible. We show that with the "truncating windows" analysis and some conclusions obtained with SPF, the spectrum expansion caused by non-Lambertian reflections and occlusions can be quantatively estimated, even when the scene geometry is not explicitly known. Given the spectrum of IBR, we also study how to sample IBR data more efficiently. Our analysis is based on the generalized periodic sampling theory with arbitrary geometry. We show that the sampling efficiency can be up to twice of that when we use rectangular sampling. The advantages and disadvantages of generalized periodic sampling for IBR are also discussed. [ABSTRACT FROM AUTHOR]

In this paper, we survey the techniques for image-based rendering (IBR) and for compressing image-based representations. Unlike traditional three-dimensional (3-D) computer graphics, in which 3-D geometry of the scene is known, IBR techniques render novel views directly from input images. IBR techniques ran be classified into three categories according to how much geometric information is used: rendering without geometry, rendering with implicit geometry (i.e., correspondence), and rendering with explicit geometry (either with approximate or accurate geometry). We discuss the characteristics of these categories and their representative techniques.IBR techniques demonstrate a surprising diverse range in their extent of use of images and geometry in representing 3-D scenes. We explore the issues in trading off the use of images and geometry by revisiting plenoptic-sampling analysis and the notions of view dependency and geometric proxies. Finally, we highlight compression techniques specifically designed for image-based representations. Such compression techniques are important in making IBR techniques practical. [ABSTRACT FROM AUTHOR]

Binary image processing is a powerful tool in many image and video applications. A reconfigurable processor is presented for binary image processing in this paper. The processor's architecture is a combination of a reconfigurable binary processing module, input and output image control units, and peripheral circuits. The reconfigurable binary processing module, which consists of mixed-grained reconfigurable binary compute units and output control logic, performs binary image processing operations, especially mathematical morphology operations, and implements related algorithms more than 200 f/s for a 1024\,\times\,1024 image. The periphery circuits control the whole image processing and dynamic reconfiguration process. The processor is implemented on an EP2S180 field-programmable gate array. Synthesis results show that the presented processor can deliver 60.72 GOPS and 23.72 GOPS/mm^2 at a 220-MHz system clock in the SMIC 0.18-\mum CMOS process. The simulation and experimental results demonstrate that the processor is suitable for real-time binary image processing applications. [ABSTRACT FROM PUBLISHER]

The disparity estimation problem is commonly solved using graph cut (GC) methods, in which the disparity assignment problem is transformed to one of minimizing global energy function. Although such an approach yields an accurate disparity map, the computational cost is relatively high. Accordingly, this paper proposes a hierarchical bilateral disparity structure (HBDS) algorithm in which the efficiency of the GC method is improved without any loss in the disparity estimation performance by dividing all the disparity levels within the stereo image hierarchically into a series of bilateral disparity structures of increasing fineness. To address the well-known foreground fattening effect, a disparity refinement process is proposed comprising a fattening foreground region detection procedure followed by a disparity recovery process. The efficiency and accuracy of the HBDS-based GC algorithm are compared with those of the conventional GC method using benchmark stereo images selected from the Middlebury dataset. In addition, the general applicability of the proposed approach is demonstrated using several real-world stereo images. [ABSTRACT FROM PUBLISHER]

Moments extraction from high resolution images in real time may require a large amount of hardware resources. Using a direct method may involve a critically high operating frequency. This paper presents two improved digital-filter based moment accelerators, as exemplified by a Tchebichef moments computation engine, to introduce features that contribute to an area-efficient and timing-efficient accelerator design. The design of the accelerators invariably consists of two on-chip units: the digital filter and the matrix multiplication units. Among the features introduced are: a data-shifting means, a filter load distribution method, a reduced set of column filters, sectioned left shifters, a double-line buffer, time-multiplexed and pipelined matrix multiplication sections, and multichip amenable features. A total of 98 frames of test data from high definition videos, real and synthetic images are used in the functional tests. The single-chip field-programmable gate array implementation results show the successful realizations of accelerators capable of moment computations of (31, 31) orders, at 50 frames of 1920\,\times\,1080 8-bit pixels per second, and (63, 63) orders, at 30 frames of 512\,\times\,512 pixels per second. These performances have exceeded that of existing multichip and multiplatform designs. [ABSTRACT FROM AUTHOR]

Today, many people in the world without any (or with little) knowledge about video recording, thanks to the widespread use of mobile devices (personal digital assistants, mobile phones, etc.), take videos. However, the unwanted movements of their hands typically blur and introduce disturbing jerkiness in the recorded sequences. Many video stabilization techniques have been hence developed with different performances but only fast strategies can be implemented on embedded devices. A fundamental issue is the overall robustness with respect to different scene contents (indoor, outdoor, etc.) and conditions (illumination changes, moving objects, etc.). In this paper, we propose a fast and robust image alignment algorithm for video stabilization purposes. Our contribution is twofold: a fast and accurate block-based local motion estimator together with a robust alignment algorithm based on voting. Experimental results confirm the effectiveness of both local and global motion estimators. [ABSTRACT FROM AUTHOR]