Search

The problems in computer vision of finding the global correspondences across a set of images can be formulated as a multiple graph matching problem consisting of pairwise matching problems. In the multiple graph matching problem, matching consistency is as important as matching accuracy for preventing the contrariety among matched results. Unfortunately, since the majority of conventional pairwise matching methods only approximate the original graph matching problem owing to its computational complexity, a framework that separately matches each graph pair could generate inconsistent results in practical environments. In this paper, we propose a novel multiple graph matching method based on the second-order consistency concept, which simultaneously considers the matching information of all possible graph pairs. We reformulate the multiple graph matching problem to encourage second-order consistency and design an iterative optimization framework. In our experiments, the proposed method outperforms the state-of-the-art methods in terms of both consistency and accuracy.

A robust algorithm is proposed for tracking a target object in dynamic conditions including motion blurs, illumination changes, pose variations, and occlusions. To cope with these challenging factors, multiple trackers based on different feature representations are integrated within a probabilistic framework. Each view of the proposed multiview (multi-channel) feature learning algorithm is concerned with one particular feature representation of a target object from which a tracker is developed with different levels of reliability. With the multiple trackers, the proposed algorithm exploits tracker interaction and selection for robust tracking performance. In the tracker interaction, a transition probability matrix is used to estimate dependencies between trackers. Multiple trackers communicate with each other by sharing information of sample distributions. The tracker selection process determines the most reliable tracker with the highest probability. To account for object appearance changes, the transition probability matrix and tracker probability are updated in a recursive Bayesian framework by reflecting the tracker reliability measured by a robust tracker likelihood function that learns to account for both transient and stable appearance changes. Experimental results on benchmark datasets demonstrate that the proposed interacting multiview algorithm performs robustly and favorably against state-of-the-art methods in terms of several quantitative metrics.

This paper presents a practical framework for occlusion-aware augmented reality application using visual-inertial RGB-D SLAM. First, an efficient visual SLAM framework with map merging based relocalization is introduced. When the pose estimation fails, a new environment map is generated. Then, a map merging is performed to merge the current and previous environment maps if a loop closure is detected. The framework is then integrated with the inertial information to solve the missing environment map problem. Camera pose is approximated using the angular velocity and translational acceleration value when the pose estimation fails. Experimental results show that the proposed method can perform well in the presence of missing pose. Finally, an occlusion-aware augmented reality application is built over the SLAM framework.

Recent achievement of the learning-based classification leads to the noticeable performance improvement in automatic polyp detection. Here, building large good datasets is very crucial for learning a reliable detector. However, it is practically challenging due to the diversity of polyp types, expensive inspection, and labor-intensive labeling tasks. For this reason, the polyp datasets usually tend to be imbalanced, i.e., the number of non-polyp samples is much larger than that of polyp samples, and learning with those imbalanced datasets results in a detector biased toward a non-polyp class. In this paper, we propose a data sampling-based boosting framework to learn an unbiased polyp detector from the imbalanced datasets. In our learning scheme, we learn multiple weak classifiers with the datasets rebalanced by up/down sampling, and generate a polyp detector by combining them. In addition, for enhancing discriminability between polyps and non-polyps that have similar appearances, we propose an effective feature learning method using partial least square analysis, and use it for learning compact and discriminative features. Experimental results using challenging datasets show obvious performance improvement over other detectors. We further prove effectiveness and usefulness of the proposed methods with extensive evaluation.

For stereo matching, PatchMatch belief propagation (PMBP) gives an efficient way of inferencing continuous labels on the Markov random field. Nevertheless, it still requires considerable time when the resolution of input images is high. To handle high-resolution images, a two-step stereo method is proposed that efficiently exploits PMBP by depth upsampling. In the first step, PMBP is conducted on the random field whose nodes correspond to the downsampled pixels from an input image. As a result, accurate low-resolution disparity maps are efficiently obtained by taking advantage of PMBP. In the second step, the low-resolution disparity map is upsampled while considering depth boundaries and sub-pixel accuracy. Experimental results show that the proposed method provides more accurate disparity maps than the original PMBP while reducing computation time remarkably.

When there exists camera motion as well as scene motion, the position and disparity of a pixel in an image sequence temporally vary with time. Such image motion (IM) and temporal disparity variation (TDV) degrade the performance of spatiotemporal stereo matching. In this Letter, a robust spatiotemporal similarity measure that addresses IM and TDV is proposed. To this end, an irregular spatiotemporal window whose temporal windows are located by a TDV value and optical flows is designed. In addition, a spatiotemporal guided filter-based aggregation technique using temporal weights based on temporal proximity and flow reliability is presented. To handle a large number of labels effectively, a search range reduction method for finding a probable label set is presented. Experimental results show that the proposed method yields consistent and accurate disparity maps under IM and TDV.

To watch 360�� videos on normal 2D displays, we need to project the selected part of the 360�� image onto the 2D display plane. In this paper, we propose a fully-automated framework for generating content-aware 2D normal-view perspective videos from 360�� videos. Especially, we focus on the projection step preserving important image contents and reducing image distortion. Basically, our projection method is based on Pannini projection model. At first, the salient contents such as linear structures and salient regions in the image are preserved by optimizing the single Panini projection model. Then, the multiple Panini projection models at salient regions are interpolated to suppress image distortion globally. Finally, the temporal consistency for image projection is enforced for producing temporally stable normal-view videos. Our proposed projection method does not require any user-interaction and is much faster than previous content-preserving methods. It can be applied to not only images but also videos taking the temporal consistency of projection into account. Experiments on various 360�� videos show the superiority of the proposed projection method quantitatively and qualitatively., Accepted to International Conference on Computer Vision (ICCV), 2017

Due to the recent explosion in various forms of 3D content, the evaluation of such content from a neuroscience perspective is quite interesting. However, existing investigations of cortical oscillatory responses in stereoscopic depth perception are quite rare. Therefore, we investigated spatiotemporal and spatio-temporo-spectral features at four different stereoscopic depths within the comfort zone. We adopted a simultaneous EEG/MEG acquisition technique to collect the oscillatory responses of eight participants. We defined subject-specific retinal disparities and designed a single trial-based stereoscopic viewing experimental paradigm. In the group analysis, we observed that, as the depth increased from Level 1 to Level 3, there was a time-locked increase in the N200 component in MEG and the P300 component in EEG in the occipital and parietal areas, respectively. In addition, initial alpha and beta event-related desynchronizations (ERD) were observed at approximately 500 to 1000 msec, while theta, alpha, and beta event-related synchronizations (ERS) appeared at approximately 1000 to 2000 ms. Interestingly, there was a saturation point in the increase in cognitive responses, including N200, P300, and alpha ERD, even when the depth increased only within the comfort zone. Meanwhile, the magnitude of low beta ERD decreased in the dorsal pathway as depth increased. From these findings, we concluded that cognitive responses are likely to become saturated in the visual comfort zone, while perceptual load may increase with depth.

For 3D object detection and pose estimation, it is crucial to extract distinctive and representative features of the objects and describe them efficiently. Therefore, a large number of 3D feature descriptors has been developed. Among these, Point Feature Histogram RGB (PFHRGB) has been evaluated as showing the best performance for 3D object and category recognition. However, this descriptor is vulnerable to point density variation and produces many false correspondences accordingly. In this paper, we tackle this problem and propose an algorithm to find the correct correspondences under the point density variation. Experimental results show that the proposed method is promising for 3D object detection and pose estimation under the point density variation.

When computing a depth map sequence of a stereo image sequence, the temporal consistency of computed depth maps is a very important factor along with the accuracy. In this paper, we propose a new similarity measure for spatiotemporal stereo matching aiming at producing temporally consistent depth maps from a stereo image sequence. To enforce the temporal consistency in a spatiotemporal similarity measure, we assign adaptive support weights to pixels in a spatiotemporal window and define the four-dimensional support region in consideration of the motion and depth variation along the time. In addition, we model the support weight to be less sensitive to illumination variation. The similarity is computed simply by comparing two support regions with computed support weights. The proposed similarity measure truly improves the performance of stereo matching both in the accuracy and in the consistency aspects.

The detection of dynamic points on a moving platform is an important task to avoid a potential collision. However, it is difficult to detect dynamic points using only two frames, especially when various input data such as ego-motion, disparity map, and optical flow are noisy for computing the motion of points. In this paper, we propose a supervised learning-based approach to detect dynamic points in consideration of noisy input data. First of all, to consider depth ambiguity that proportionally increases according to the distance to the ego-vehicle, we divide the XZ-plane (bird-eye view) into several subregions. Then, we train a random forest for each subregion by constructing motion vectors computed based on two motion metrics. Here, in order to reduce errors of the input data, the motion vectors are filtered based on a pairwise planarity check and then filtered motion vectors are used for training. In the experiments, the proposed method is verified by comparing the detection performance with that of previous approaches on the KITTI dataset.

In this paper, we propose a novel method to jointly solve scene layout estimation and global registration problems for accurate indoor 3D reconstruction. Given a sequence of range data, we first build a set of scene fragments using KinectFusion and register them through pose graph optimization. Afterwards, we alternate between layout estimation and layout-based global registration processes in iterative fashion to complement each other. We extract the scene layout through hierarchical agglomerative clustering and energy-based multi-model fitting in consideration of noisy measurements. Having the estimated scene layout in one hand, we register all the range data through the global iterative closest point algorithm where the positions of 3D points that belong to the layout such as walls and a ceiling are constrained to be close to the layout. We experimentally verify the proposed method with the publicly available synthetic and real-world datasets in both quantitative and qualitative ways., Accepted to 2017 IEEE International Conference on Computer Vision (ICCV)

Multi-attributed graph matching is a problem of finding correspondences between two sets of data while considering their complex properties described in multiple attributes. However, the information of multiple attributes is likely to be oversimplified during a process that makes an integrated attribute, and this degrades the matching accuracy. For that reason, a multi-layer graph structure-based algorithm has been proposed recently. It can effectively avoid the problem by separating attributes into multiple layers. Nonetheless, there are several remaining issues such as a scalability problem caused by the huge matrix to describe the multi-layer structure and a back-projection problem caused by the continuous relaxation of the quadratic assignment problem. In this work, we propose a novel multi-attributed graph matching algorithm based on the multi-layer graph factorization. We reformulate the problem to be solved with several small matrices that are obtained by factorizing the multi-layer structure. Then, we solve the problem using a convex-concave relaxation procedure for the multi-layer structure. The proposed algorithm exhibits better performance than state-of-the-art algorithms based on the single-layer structure., 10 pages, 4 figures, conference submitted

Person re-identification is the task of recognizing or identifying a person across multiple views in multi-camera networks. Although there has been much progress in person re-identification, person re-identification in large-scale multi-camera networks still remains a challenging task because of the large spatio-temporal uncertainty and high complexity due to a large number of cameras and people. To handle these difficulties, additional information such as camera network topology should be provided, which is also difficult to automatically estimate, unfortunately. In this study, we propose a unified framework which jointly solves both person re-identification and camera network topology inference problems with minimal prior knowledge about the environments. The proposed framework takes general multi-camera network environments into account and can be applied to online person re-identification in large-scale multi-camera networks. In addition, to effectively show the superiority of the proposed framework, we provide a new person re-identification dataset with full annotations, named SLP, captured in the multi-camera network consisting of nine non-overlapping cameras. Experimental results using our person re-identification and public datasets show that the proposed methods are promising for both person re-identification and camera topology inference tasks., Comment: 14 pages, 14 figures, 6 tables

We address the correspondence search problem among multiple graphs with complex properties while considering the matching consistency. We describe each pair of graphs by combining multiple attributes, then jointly match them in a unified framework. The main contribution of this paper is twofold. First, we formulate the global correspondence search problem of multi-attributed graphs by utilizing a set of multi-layer structures. The proposed formulation describes each pair of graphs as a multi-layer structure, and jointly considers whole matching pairs. Second, we propose a robust multiple graph matching method based on the multi-layer random walks framework. The proposed framework synchronizes movements of random walkers, and leads them to consistent matching candidates. In our extensive experiments, the proposed method exhibits robust and accurate performance over the state-of-the-art multiple graph matching algorithms.

Person re-identification is the problem of recognizing people across different images or videos with non-overlapping views. Although there has been much progress in person re-identification over the last decade, it remains a challenging task because appearances of people can seem extremely different across diverse camera viewpoints and person poses. In this paper, we propose a novel framework for person re-identification by analyzing camera viewpoints and person poses in a so-called Pose-aware Multi-shot Matching (PaMM), which robustly estimates people's poses and efficiently conducts multi-shot matching based on pose information. Experimental results using public person re-identification datasets show that the proposed methods outperform state-of-the-art methods and are promising for person re-identification from diverse viewpoints and pose variances., Comment: 12 pages, 12 figures, 4 tables

Keyframe-based camera tracking methods can reduce error accumulation in that they reduce the number of camera poses to be estimated by selecting a set of keyframes from an image sequence. In this paper, we propose a novel Bayesian filtering framework for keyframe-based camera tracking and 3D mapping. Our Bayesian filtering enables an effective estimation of keyframe poses using all measurements obtained at non-keyframe locations, which improves the accuracy of the estimated path. In addition, we discuss the independence problem between the process noise and the measurement noise when employing vision-based motion estimation approaches for the process model, and we present a method of ensuring independence by dividing the measurements obtained from a single sensor into two sets which are exclusively used for the process and measurement models. We demonstrate the performance of the proposed approach in terms of the consistency of the global map and the accuracy of the estimated path.

In this paper, we consider a multiobject tracking problem in complex scenes. Unlike batch tracking systems using detections of the entire sequence, we propose a novel online multiobject tracking system in order to build tracks sequentially using online provided detections. To track objects robustly even under frequent occlusions, the proposed system consists of three main parts: 1) visual tracking with a novel data association with a track existence probability by associating online detections with the corresponding tracks under partial occlusions; 2) track management to associate terminated tracks for linking tracks fragmented by long-term occlusions; and 3) online model learning to generate discriminative appearance models for successful associations in other two parts. Experimental results using challenging public data sets show the obvious performance improvement of the proposed system, compared with other state-of-the-art tracking systems. Furthermore, extensive performance analysis of the three main parts demonstrates effects and usefulness of the each component for multiobject tracking.

Depth map up-sampling methods have achieved remarkable improvement by exploiting sensor fusion techniques where they assume that the depth map discontinuities and image edges coincide, and the depth values of the temporal neighbors are stable during time variation. However, inherent noise of depth data acquired by active range sensors often violates these assumptions, and results in undesirable error propagation. To alleviate the error propagation, this letter presents a new adaptive supporting method of spatially-temporally neighboring samples. On the basis of a spatial-temporal Markov random field model, the weight coefficients of the smoothness terms are adaptively computed according to the reliability of neighboring samples. The experiments show that the proposed method outperforms the previous works in terms of quantitative and qualitative criteria.

In this paper, we present a method for obtaining a highquality 3D depth. The advantages of active pattern projection and passive stereo matching are combined and a system is established. A diffractive optical element (DOE) is developed to project the active pattern. Cross guidance (CG) and auto guidance (AG) are proposed to perform the passive stereo matching in a stereo image in which a DOE pattern is projected. When obtaining the image, the CG emits a DOE pattern periodically and consecutively receives the original and pattern images. In addition, stereo matching is performed using these images. The AG projects the DOE pattern continuously. It conducts cost aggregation, and the image is restored through the process of removing the pattern from the pattern image. The ground truth is generated to estimate the optimal parameter among various stereo matching algorithms. Using the ground truth, the optimal parameter is estimated and the cost computation and aggregation algorithm are selected. The depth is calculated and bad-pixel errors make up 4.45% of the non-occlusion area.

The unscented sequential Monte Carlo probability hypothesis density (USMC-PHD) filter has been proposed to improve the accuracy performance of the bootstrap SMC-PHD filter in cluttered environments. However, the USMC-PHD filter suffers from heavy computational complexity because the unscented information filter is assigned for every particle to approximate an importance sampling function. In this paper, we propose a Gaussian mixture form of the importance sampling function for the SMC-PHD filter to considerably reduce the computational complexity without performance degradation. Simulation results support that the proposed importance sampling function is effective in computational aspects compared with variants of SMC-PHD filters and competitive to the USMC-PHD filter in accuracy.

In visual object tracking, exploiting correlation filters to track the target of interest has been flourished, and, by adopting a circulant form of an image or feature descriptors jointed with the convolution theorem, these correlation filter trackers surpass many of the previous state-of-the-art trackers in both tracking speed and stability. Nevertheless, when the appearance of the target object abruptly changes due to occlusion, background cluttering, or viewpoint variation, even the aforementioned correlation filter trackers still tend to fail to compute a reliable correlation output. Concerned with this problem, we propose a method that observes the locational drift of the correlation peak from the desired location. Utilizing this information, we restrict the searching range of the correlation peak to increase the accuracy of the tracker. We verify the performance of the proposed tracker by using 2014 Visual Object Tracking Challenge benchmark dataset.

Person re-identification is the problem of recognizing people across images or videos from non-overlapping views. Although there has been much progress in person re-identification for the last decade, it still remains a challenging task because of severe appearance changes of a person due to diverse camera viewpoints and person poses. In this paper, we propose a novel framework for person reidentification by analyzing camera viewpoints and person poses, so-called Pose-aware Multi-shot Matching (PaMM), which robustly estimates target poses and efficiently conducts multi-shot matching based on the target pose information. Experimental results using public person reidentification datasets show that the proposed methods are promising for person re-identification under diverse viewpoints and pose variances.

Multi-object tracking (MOT) becomes more challenging when objects of interest have similar appearances. In that case, the motion cues are particularly useful for discriminating multiple objects. However, for online 2D MOT in scenes acquired from moving cameras, observable motion cues are complicated by global camera movements and thus not always smooth or predictable. To deal with such unexpected camera motion for online 2D MOT, a structural motion constraint between objects has been utilized thanks to its robustness to camera motion. In this paper, we propose a new data association method that effectively exploits structural motion constraints in the presence of large camera motion. In addition, to further improve the robustness of data association against mis-detections and false positives, a novel event aggregation approach is developed to integrate structural constraints in assignment costs for online MOT. Experimental results on a large number of datasets demonstrate the effectiveness of the proposed algorithm for online 2D MOT.

In this paper, we propose a novel implementation of the probability hypothesis density (PHD) filter based on the sequential Monte Carlo (SMC) method called SMC-PHD filter. The SMC-PHD filter is analogous to the sequential importance sampling which generates samples using an importance sampling (IS) function. Even though this filter permits general class of IS density function, many previous implementations have simply used the state transition density function. However, this approach leads to a degeneracy problem and renders the filter inefficient. Thus, we propose a novel IS function for the SMC-PHD filter using a combination of an unscented information filter and a gating technique. Further, we use measurement-driven birth target intensities because they are more efficient and accurate than selecting birth targets selected using arbitrary or expected mean target states. The performance of the SMC-PHD filter with the proposed IS function was subsequently evaluated through a simulation and it was shown to outperform the standard SMC-PHD filter and recently proposed auxiliary PHD filter.

The paper experimentally verifies the performance of Terrain-Aided Navigation (TAN) using an interferometric radio altimeter, which is recently used due to its accuracy. First, we propose a TAN system that utilizes an interferometric radio altimeter as a measurement system. Second, we implement extended Kalman filter, unscented Kalman filter, and particle filter to evaluate the performance of TAN according to the selection of filters and the difference of environments.

The examination of the flora of the Sukpo-Naesujun section of the Ulleung forest trail revealed a total of 291 taxa, including 85 families, 211 genera, 246 species, 41 varieties, two forms, and two subspecies. In the area, the endangered plant species designated by the Ministry of Environment was Trillium tschonoskii, and rare or endangered plants designated by the Korea Forest Service were revealed to be 10 species, including Tsuga sieboldii, most of which were found around the trail. Campanula takesimana and Allium victorialis, in particular, are exposed to a threat of being collected for ornamental and eating purposes, respectively. The Korean endemic species in the area were 12 species including Fagus engleriana. In the area there were 50 floristic special plant species, the survey revealed 17 taxa of grade I, three taxa of grade II, nine taxa of grade III, 19 taxa of grade IV, and two taxa of grade V. Among them, Lilium hansonii and Trillium tschonoskii from grades IV and V are located near the trail, leading to concerns about damage by mountain climbers or residents. The naturalized plants in the area were a total of 18 taxa, including 6 families, 14 genera, 16 species, and one variety. Moreover, 25.4% of the overall naturalized plants in Ulleung-do appeared in the Ulleung forest trail.

In this paper, we propose new symmetric cost functions for global stereo methods. We first present a symmetric data cost function for the likelihood and then propose a symmetric discontinuity cost function for the prior in the MRF model for stereo. In defining cost function, both the reference image and the target image are taken into account to improve performance without modeling half-occluded pixels explicitly. The performance improvement of stereo matching due to the proposed symmetric cost functions is verified by applying the proposed symmetric cost functions to the belief propagation (BP) based stereo method.

A low-cost inertial measurement unit (IMU) and a rolling shutter camera form a conventional device configuration for localization of a mobile platform due to their complementary properties and low costs. This paper proposes a new calibration method that jointly estimates calibration and noise parameters of the low-cost IMU and the rolling shutter camera for effective sensor fusion in which accurate sensor calibration is very critical. Based on the graybox system identification, the proposed method estimates unknown noise density so that we can minimize calibration error and its covariance by using the unscented Kalman filter. Then, we refine the estimated calibration parameters with the estimated noise density in batch manner. Experimental results on synthetic and real data demonstrate the accuracy and stability of the proposed method and show that the proposed method provides consistent results even with unknown noise density of the IMU. Furthermore, a real experiment using a commercial smartphone validates the performance of the proposed calibration method in off-the-shelf devices.

We exploit the advantages of confidence measures to improve the robustness and accuracy of the weighted median filter (WMF). The conventional WMF fails to find a correct solution as the proportion of unreliable disparity values increases. Moreover, correct disparity values can be easily affected by unreliable pixels. To alleviate this problem, we consider the reliability of pixels as an additional weighting strategy where the reliability is computed by conventional confidence measures. We experimentally verified the proposed method on the Middlebury dataset in terms of bad pixel rates with various filtering methods.

Online multi-object tracking with a single moving camera is a challenging problem as the assumptions of 2D conventional motion models (e.g., first or second order models) in the image coordinate no longer hold because of global camera motion. In this paper, we consider motion context from multiple objects which describes the relative movement between objects and construct a Relative Motion Network (RMN) to factor out the effects of unexpected camera motion for robust tracking. The RMN consists of multiple relative motion models that describe spatial relations between objects, thereby facilitating robust prediction and data association for accurate tracking under arbitrary camera movements. The RMN can be incorporated into various multi-object tracking frameworks and we demonstrate its effectiveness with one tracking framework based on a Bayesian filter. Experiments on benchmark datasets show that online multi-object tracking performance can be better achieved by the proposed method.

We present a new window-based method for correspondence search using varying support-weights. We adjust the support-weights of the pixels in a given support window based on color similarity and geometric proximity to reduce the image ambiguity. Our method outperforms other local methods on standard stereo benchmarks.

This paper describes a sonic polygonal multiple reflection range sensor (SPMRS), which uses multiple reflection properties usually ignored in ultrasonic sensors as disturbances or noises. Targets such as a plane, corner, edge, or cylinder in indoor environments can easily be detected by the multiple reflection patterns obtained with a SPMRS system. Target classification and feature data extraction, such as distance and azimuth to the target, are computed simultaneously by considering the geometrical relationships between the detected targets, and finally the environment model is generated by refining the detected targets. In addition, the narrow field of view of a sonar range sensor is increased and the scanning time is reduced by active motion of the SPMRS stepping servomechanism.

Applications for tracking multiple objects in an image sequence are frequently challenged by various uncertainties, such as occlusion, misdetection, and abrupt camera motion. In practical environments, these uncertainties may occur simultaneously and with no pattern so that they must be jointly considered to achieve reliable tracking. We propose a two-step online multi-object tracking framework that incorporates a confidence-aided relative motion network (RMN) to jointly consider various difficulties. Because of the framework’s two-step data association process and the similarity function using RMNs, the proposed method achieves robust performance in the presence of most kinds of uncertainties. In our experiments, the proposed method exhibits a very robust and efficient performance compared with other state-of-the-art algorithms.

Proposed is a novel key-frame selection method for the video-based structure-from-motion technique. This method consists of two main steps. First, unreliable features are eliminated by analysing the lifetimes of the tracked features. Next, an optimal set of key-frames is selected considering sufficient baselines between key-frames, data redundancy, and the avoidance of degeneracy. The proposed method is experimentally verified in terms of accuracy and stability for various video sequences.

Online multi-object tracking aims at producing complete tracks of multiple objects using the information accumulated up to the present moment. It still remains a difficult problem in complex scenes, because of frequent occlusion by clutter or other objects, similar appearances of different objects, and other factors. In this paper, we propose a robust online multi-object tracking method that can handle these difficulties effectively. We first propose the tracklet confidence using the detectability and continuity of a tracklet, and formulate a multi-object tracking problem based on the tracklet confidence. The multi-object tracking problem is then solved by associating tracklets in different ways according to their confidence values. Based on this strategy, tracklets sequentially grow with online-provided detections, and fragmented tracklets are linked up with others without any iterative and expensive associations. Here, for reliable association between tracklets and detections, we also propose a novel online learning method using an incremental linear discriminant analysis for discriminating the appearances of objects. By exploiting the proposed learning method, tracklet association can be successfully achieved even under severe occlusion. Experiments with challenging public datasets show distinct performance improvement over other batch and online tracking methods.

We present a simple artificial landmark model and a robust landmark tracking alg orithm for mobile robot localization. The landmark model, consisting of symmetric and repetitive color patches, produces color histograms that are invariant under the geometric and photometric distortions. A stochastic approach based on the CONDENSATION tracks the landmark model robustly even under the varying illumination conditions. After the landmark detection, relative position of the mobile robot to the landmark is calculated. Experimental results show that the proposed landmark model is effective and can be detected and tracked in a cluttered scenerobustly. The tracked single landmark is enough to detennine the position of the robot.

An accurate stereo matching method developed by exploiting the two techniques, discrete-coded structured-light projection and image-guided cost volume filtering, is proposed. The former increases the distinctiveness of pixels by projecting a discrete pattern to the scene, and the latter helps to recover accurate object boundaries. In addition, a previous fast cost volume filtering approach is extended to better preserve slanted surfaces, and a suitable post-processing algorithm is also suggested for the proposed method. The performance of the proposed method is experimentally verified by comparing the results with those of other algorithms qualitatively and quantitatively in an indoor environment.

When there exists camera and scene motion, the disparity of a pixel temporally varies as time goes on. Such temporal disparity variation (TDV) degrades the performance of spatiotemporal stereo matching. In this paper, we devise a robust similarity measure against TDV, and a suitable optimization technique for the proposed measure. We first design the window-based matching cost to evaluate the similarity between pixels for given disparity and a TDV value. We also present the improved spatiotemporal guided-filter-based aggregation technique to gather match costs with temporal weights. The disparity and TDV maps are then obtained by the global optimization. Here, to handle the large number of labels (disparity levels × TDV levels), we use dual-layer belief propagation that requires less computation and memory while producing comparable results with belief propagation using a single layer. Experimental results show the proposed method yields consistent and accurate disparity maps under the TDV.

In this paper, we propose a novel multi-object tracking method to track unknown number of objects with a single camera system. We design the tracking method via probability hypothesis density (PHD) filtering which considers multiple object states and their observations as random finite sets (RFSs). The PHD filter is capable of rejecting clutters, handling object appearances and disappearances, and estimating the trajectories of multiple objects in a unified framework. Although the PHD filter is robust to cluttered environment, it is vulnerable to missed detections. For this reason, we include local observations in an RFS of observation model. Local observations are locally generated near the individual tracks by using on-line trained local detector. The main purpose of the local observation is to handle the missed detections and to provide identity (label information) to each object in filtering procedure. The experimental results show that the proposed method robustly tracks multiple objects under practical situations.

Visual discomfort induced by inharmonious human 3D perceptions between the real world and a display is encountered most commonly in stereoscopic displays. Although various state-of-the-art 3D technologies have reduced the gaps, these delicate differences yield eye fatigue during adaptation of the human brain to artificial 3D information in displays. Therefore, the assessment of visual discomfort, either in producing or reproducing 3D content, is essential to improve viewers' quality of experience. In this paper, we used EEG to investigate the feasibility of visual discomfort assessment on stereo images. To find features of brain-waves representing 3D visual discomfort, we designed 3D stereo image experiments and collected EEG data for several subjects. We found that spectral attenuation of alpha and beta bands over the sensorimotor area and temporal features detected 2 to 4 seconds after onset could be strong indicators of visual discomfort.

A new method to up-sample low-resolution depth maps to high quality and high resolution is proposed. Range sensors such as time-of-flight cameras yield low-resolution depth maps and the output includes heavy noise at the edges of objects. Recently, many techniques have been proposed to up-sample low-resolution depth maps. However, there is no effective countermeasure to the edge noise problem. The proposed up-sampling method is based on Markov random fields and addresses this edge noise using newly designed confidence weights. The performance of our method is evaluated using error rate and mean absolute error through comparison with existing methods, and results show that the proposed method outperforms conventional methods.

This paper presents a tone compensation method between images to make a seamless panoramic image. Different camera settings of input images, including white-balance, exposure time, and f-stops, affect the overall color tone of a resultant panoramic image. Although numerous methods have been proposed to deal with such color variations for seamless image stitching, most of them do not properly consider the dynamic scene in which different scene contents exist in input images. In this paper, we propose an efficient method that takes dynamic scene contents into account for compensating color tone difference. The proposed approach consists of three steps. First, we compensate the color tone difference by using the linear color transform with robust local features. Second, we filter out dynamic objects (i.e., dynamic scene contents) by measuring similarity between the linear transformed image and the reference image. Finally, we precisely correct the color variation with detected consistent regions only. The qualitative evaluation shows superior or competitive results compared to commercially available products.

In this paper, a robust visual tracking method is proposed to track an object in dynamic conditions that include motion blur, illumination changes, pose variations, and occlusions. To cope with these challenges, multiple trackers with different feature descriptors are utilized, and each of which shows different level of robustness to certain changes in an object's appearance. To fuse these independent trackers, we propose two configurations, tracker selection and interaction. The tracker interaction is achieved based on a transition probability matrix (TPM) in a probabilistic manner. The tracker selection extracts one tracking result from among multiple tracker outputs by choosing the tracker that has the highest tracker probability. According to various changes in an object's appearance, the TPM and tracker probability are updated in a recursive Bayesian form by evaluating each tracker's reliability, which is measured by a robust tracker likelihood function (TLF). When the tracking in each frame is completed, the estimated object's state is obtained and fed into the reference update via the proposed learning strategy, which retains the robustness and adaptability of the TLF and multiple trackers. The experimental results demonstrate that our proposed method is robust in various benchmark scenarios.

Local feature-based object recognition methods recognize learned objects by unordered local feature matching followed by verification. However, the matching between unordered feature sets might be ambiguous as the number of objects increases, because multiple similar features can be observed in different objects. In this context, we present a new method for textured object recognition based on relational information between local features. To efficiently reduce ambiguity, we represent objects using the Attributed Relational Graph. Robust object recognition is achieved by the inexact graph matching. Here, we propose a new method for building graphs and define robust attributes for nodes and edges of the graph, which are the most important factors in the graphbased object representation, and also propose a cost function for graph matching. Dependent on the proposed attributes, the proposed framework can be applied to both single-image-based and stereo-image-based object recognition.