Search

A proposed framework using spectral and spatial information is introduced for neural net multisensor data fusion. This consists of a set of independent-sensor neural nets, one for each sensor (type of data), coupled to a fusion net. The neural net of each sensor is trained from a representative data set of the particular sensor to map to a hypothesis space output. The decision outputs from the sensor nets are used to train the fusion net to an overall decision. During the initial processing, three-dimensional (3-D) point cloud data (PCD) are segmented using a multidimensional mean-shift algorithm into clustered objects. Concurrently, multiband spectral imagery data (multispectral or hyperspectral) are spectrally segmented by the stochastic expectation–maximization into a cluster map containing (spectral-based) pixel classes. For the proposed sensor fusion, spatial detections and spectral detections complement each other. They are fused into final detections by a cascaded neural network, which consists of two levels of neural nets. The success of the approach in utilizing sensor synergism for an enhanced classification is demonstrated for the specific case of classifying hyperspectral imagery and PCD extracted from LIDAR, obtained from an airborne data collection over the campus of University of Southern Mississippi, Gulfport, Mississippi.

Optical imaging systems are often degraded by scattering due to atmospheric particles such as haze, fog, and mist. According to the physical model of atmospheric scattering, two major factors that affect the effect of fog removal are discussed. By using joint probability density definition and the proposed joint mean shift filtering algorithm, discontinuity preserving smoothing and image segmentation are addressed in the sequel to refine the atmospheric veil and estimate the atmospheric light, which overcomes the inherent limitations of the state-of-the-art methods. Simulation results on a variety of outdoor foggy images demonstrate that the proposed method achieves short computation time and good restoration for visibility and color fidelity. An embedded vision enhanced system based on a DSP is designed and runs the proposed algorithm at 15 fps for a color frame of 720×576 pixels .

Over the past four decades, the satellite imaging sensors have acquired huge quantities of Earth- observation data. Content-based image retrieval allows for fast and effective queries of remote sensing images. Here, we take the following two issues into consideration. Firstly, different features and their combination should be chosen for different land covers. Secondly, for the block dividing strategy and the complexities of the remote sensing images, it can not effectively retrieve some small target areas scattered in multiple nontarget blocks. Aiming at the above two issues, a new region-based retrieval method with adaptive image segmentation is proposed. In order to improve the accuracy of remote sensing image segmentation, feature selection and weighing is performed by two-stage clustering, and image segmentation is accomplished based on the chosen features and mean shift procedure. Meanwhile, for the homogeneous characteristics of remote sensing land covers, a new regional representation and matching scheme are adopted to perform image retrieval. Experimental results on retrieving various land covers show that the method can avoid the impact of traditional blocking strategies, and can achieve an average percentage of 19% higher precision with the same level of recall rate, than the relevance feedback method for small target areas.

Here, we present a novel object of interest (OOI) extraction framework designed for low-frame-rate (LFR) image sequences, typically from mobile mapping systems (MMS). The proposed method integrates tracking and segmentation in a unified framework. We propose a novel object-shaped kernel-based scale-invariant mean shift algorithm to track the OOI through the LFR sequences and keep the temporal consistency. Then the well-known GrabCut approach for static image segmentation is generalized to the LFR sequences. We analyze the imaging geometry of the OOI in LFR sequences collected by the MMS and design a Kalman filter module to assist the proposed tracker. Extensive experimental results on real LFR sequences collected by VISAT™ MMS demonstrate that the proposed approach is robust to the challenges such as low frame rate, fast scaling, and large inter-frame displacement of the OOI.

We present a simple and fast integration method, which can fuse registered multiple, overlapping range images effectively while preserving the detailed features. First, normal vectors of the nodes in each range image are estimated and the registered multiple overlapping range images are combined into one point set. Due to registration errors, the normal vectors of homogeneous nodes in the overlapping areas would differ from each other. Thus, the normal vectors of the nodes are corrected iteratively by weighted averaging their neighboring normal vectors. Second, the combined point set is partitioned into a number of small clusters using the k-means clustering algorithm based on user-defined space interval. The clusters subsequently are subdivided into smaller subclusters by using the standard normal vector deviation as a measure of cluster scatter. Finally, in the cluster, the local maxima model, obtained using the mean shift clustering algorithm, is employed to represent the cluster. The proposed method is fast because it does not need to detect iteratively the overlapping areas, which is usually time consuming. In addition, because of the noisy filtering property of the mean shift clustering algorithm, the novel method can produce a smooth and watertight point surface while preserving the detailed features.

This paper proposes a robust detection and tracking method that can detect and track ship targets in cluttered forward looking infrared image sequences. The proposed method includes two major steps, tar- get detection and target tracking. Because the target's imaging size will be scaled-up gradually along with the imaging distance being reduced, the target's imaging process is analyzed in detail and the target's imag- ing size is estimated in real-time. Based on the estimation of the target's imaging size, the targets with different imaging sizes can be treated as small targets by image scaling operation and detected successfully us- ing the improved multilevel filter and iterative segmentation technique. In the tracking stage, the histogram back-projection technique, which has little effect on the movement of the camera, is applied to the mean shift procedure for real-time application. Extensive experiments demonstrate the utility and the performance of the proposed approach. C

This paper proposes a novel saliency model using multiple region-based features. The original image is initially segmented into a set of regions using the mean shift algorithm, and region merging is performed to obtain a moderate segmentation result. Then, three types of regional saliency measures are calculated using region-based features including local/global color difference, orientation difference, and spatial distribution, and they are integrated into an overall regional saliency measure for each region. Finally, the pixel-wise saliency map is generated by combining regional saliency measures with the distance-weighted color similarity between each pixel and each region. Experimental results demonstrate that our saliency model achieves an overall better saliency detection performance than previous saliency models, and the saliency maps generated using our model are more suitable for content-based applications such as salient object detection, content-aware image retargeting, and object-based image retrieval.

Graph-based dimensionality reduction methods are popular in pattern recognition and machine learning. In contrast to the manifold learning approaches, the dot product representation of graphs (DPRG) seeks a solution to dimensionality reduction by assigning vectors to each node of a graph such that the dot product of every pair of nodes approximates the similarity between them. The DPRG has many potential applications, for the reason that there is no prior assumption of the data distribution. It has been found, however, that the DPRG tends to reduce the distances of the graph nodes represented in a low-dimensional space, which in turn degrades the performance of data clustering. Motivated by this observation, we propose an extended DPRG (EDPRG) model by simply employing negative similarity values. The theoretical analysis and experiments on synthetic data show that the modification is effective in increasing between-class distances. We demonstrate the effectiveness of the EDPRG model by experiments on synthetic aperture radar (SAR) image segmentation. The proposed image segmentation method has two steps. The first one presegments the image by the mean shift algorithm. The second merges the resulting regions by means of the EDPRG model.

Representing an object with multiple image fragments or patches for target tracking in a video has proved to be able to maintain the spatial information. The major challenges in visual tracking are effectiveness and robustness. We propose a robust fragments-based tracking algorithm with adaptive feature selection. The best discriminate feature is used for tracking, which can improve tracking effectiveness. A set of likelihood images corresponding to the most discriminative features are fused to divide the object into some fragments, which can maintain the spatial information. By weighting the fragment and background colors, more robust target and candidate models are built. Given these advantages, the novel tracking algorithm can provide more accurate performance and can be directly extended to a multiple object-tracking system.

An object tracking algorithm using an adaptive Kalman filter (KF) combined with mean shift (MS) is proposed. First, the system model of KF is constructed, then the center of the object predicted by KF is used as the initial value of the MS algorithm. The searching result of MS is fed back as the measurement of the adaptive KF, and the estimate parameters of KF are adjusted by the Bhattacharyya coefficient adaptively. The proposed method has the robust ability to track a moving object in consecutive frames under certain real-world complex situations, such as a moving object disappearing partially or totally due to occlusion, fast moving objects, and sudden changes in velocity of a moving object. The experimental results demonstrate that the proposed tracking algorithm is robust and practical.

Multiple occlusion target tracking is usually a difficult problem in video surveillance. But in many cases, traditional mean shift tracking algorithms fail to track occlusion targets robustly. In this work, we focus on improving mean shift tracking algorithms to model and track all kinds of occlusion targets in video surveillance scenes. Two primary improvements on traditional mean shift tracking algorithms are proposed. First, after we determine which target the overlapping patches belong to, the nonocclusion part of each occlusion target can be obtained and applied to the tracking algorithm. Second, all the related occlusion target states are iteratively estimated one after another to eliminate the occlusion effects during the tracking process. Furthermore, the contrast experiment results show that the improved algorithm can track multiple occlusion targets, whereas traditional mean shift tracking algorithms fail.

A novel algorithm, Gaussian mean shift registration (GMSR), is proposed for multisensor dynamic bias estimation. The sufficient condition for convergence of a Gaussian mean shift procedure is given, which extends the current theorem from a strictly convex kernel to a piece-wise convex and concave kernel. The Gaussian mean shift algorithm combined with the extended Kalman filter (EKF) is implemented to estimate the dynamic bias based on the measurements from a single target, which is an iterative optimization procedure. Monte Carlo simulations show that the new algorithm has significant improvement in performance with reducing root mean square (RMS) errors compared with the minimum mean square error (MMSE) estimator, based on multiple targets and multiple frames. The proposed estimator is close to the theoretical lower bound, i.e., it is more efficient in estimating the dynamic bias than other methods.

We propose a unified approach that incorporates the mean shift-based image segmentation algorithm and the SST (shortest spanning tree)-minmax-based graph grouping method to achieve effective IR object segmentation performance amenable for real-time application. It preprocesses an image by using the mean shift algorithm to form segmented regions that can not only remove the noise, but also preserve the desirable discontinuity characteristics of the ship object. The segmented regions can then effectively represent the original image by using the graph structures, and we apply the SST-minmax method to perform merging procedure to form the final segmented regions. Due to the good discontinuity-preserving filtering characteristic, we can effectively remove the clutter disturbance of the sea background without loss of the IR ship object information, and significantly reduce the number of basic image entities. Therefore, the region merging based on SST-minmax can produce excellent segmentation performance at low computational cost due to smaller clutter disturbance and less region nodes. The superiority of the proposed method is examined and demonstrated through a large number of experiments using a real IR ship image sequence.

We present an approach that incorporates multi-information, including intensity value, spatial relation, and local standard deviation information of the pixels in target region, into kernel density estimation for constructing the kernel-based infrared (IR) target model. The incorporated information can complement each other for a target-tracking task. This constructed target model is evaluated based on the relative entropy of the two classes and is applied in a mean shift tracking system for IR target tracking to verify the effectiveness.

We propose a two-stage real-time tracking algorithm for an active camera system having pan-tilt-zoom functions. The algorithm is based on the assumption that a human head has an elliptical shape and that its model color histogram has been acquired in advance. The algo- rithm consists of two stages, a color-based convergence stage for fast and reliable tracking and a refinement stage for accurate tracking based on multimodal information. In the first color convergence stage, we roughly estimate the target position by using the mean-shift method based on the histogram similarity between the model and a candidate ellipse. To better predict the initial position for the mean shift, the global motion is compensated; to enhance reliability of the mean shift, the model histogram is appropriately updated by referring to the target his- togram in the previous frame. In the subsequent refinement stage, we refine the position and size of the ellipse obtained at the first stage by using multimodal information such as color, shape, and quasi-spatial in- formation. In particular, to quantify the quasi-spatial information, we use a spatial color histogram obtained by properly dividing the ellipse into two regions. Extensive experiments verify that the proposed algorithm robustly tracks the head, even when the subject moves quickly, the head size changes drastically, or the background has many clusters and/or distracting colors. Also, the proposed algorithm can perform real-time tracking with a processing speed of about 10 fps on a standard PC.

Classic mean-shift trackers have no integrated scale adaptation, which limits their performance in tracking variable scale objects. By analyzing the similarity of object kernel histograms, we found that the changes of object scale and position within the fixed kernel make the Bhattacharyya coefficient monotonic decreasing. The work plays a guiding role in solving scaling problems within the mean-shift framework. © 2005 Society of Photo-Optical Instrumentation Engineers.