Search

This chapter presents a sequential Monte Carlo data association algorithm for tracking a varying number of interacting objects in a dynamic scene. The algorithm is based on a computational framework that consists of a hybrid measurement process, a Monte Carlo joint probabilistic data association filter, and a particle-based belief propagation algorithm. The hybrid measurement process addresses problems associated with having a varying number of objects. It does so by mixing target-driven measurements provided by a prior model of target dynamics and data-driven measurements based on a discriminative model; the Monte Carlo joint probabilistic data association filter combats the curse of dimensionality by providing a marginal posterior distribution for each individual target; and particle-based belief propagation is used to deal with occlusions among objects.Within this framework, the learning of discriminative model, the Monte Carlo joint data association filtering, and belief propagation are realized as different levels of approximation to the ‘ideal’ generative model inherent in the problem of tracking multiple visual objects. Together, these result in an efficient sequential Monte Carlo data association algorithm. The algorithm is illustrated via the tracking of multiple pedestrians in several real-life test videos. Empirical results demonstrate the algorithm’s efficiency in a number of test situations.

Developing robust visual tracking algorithms for real-world application is a major challenge even today. In this chapter, we focus on visual tracking with data fusion using sequential Monte Carlo filtering techniques, and present a tracking framework with a four-layer probabilistic fusion. The framework consists of four layers: a cue fusion layer, a model fusion layer, a tracker fusion layer, and a sensor fusion layer, in a bottom-up fusion process. These layers are defined as follows: the cue layer fuses visual modalities via an adaptive fusion strategy, the model layer fuses prior motion information via an interactive multi-model method (IMM), the tracker layer fuses results from multiple trackers via an adaptive tracking mode hat switches between model associations, and the sensor layer fuses multiple sensors in a distributed way. Only state distributions in the input and output of each layer are required to ensure consistency of so many visual modules within the framework. Furthermore, the proposed framework is general and allows augmenting and pruning of fusion layers according to the visual environment at hand. The proposed framework is tested and satisfactory results obtained in various complex scenarios in which many existing trackers are inclined to fail.

This chapter presents an overview of the topics and major ideas of supervised learning for visual pattern classification. Two prevalent algorithms, i.e., the support vector machine (SVM) and the boosting algorithm, are briefly introduced. SVMs and boosting algorithms are two hot topics of recent research in supervised learning. SVMs improve the generalization of the learning machine by implementing the rule of structural risk minimization (SRM). It exhibits good generalization even when little training data are available for machine training. The boosting algorithm can boost a weak classifier to a strong classifier by means of the so-called classifier combination. This algorithm provides a general way for producing a classifier with high generalization capability from a great number of weak classifiers.

Multitarget tracking in video (MTTV) presents a technical challenge in video surveillance applications. In this chapter, we formulate the MTTV problem using dynamic Markov network (DMN) techniques. Our model consists of three coupled Markov random fields: (1) a field for the joint state of multiple targets; (2) a binary random process for flagging the existence of each individual target; and (3) a binary random process for flagging occlusion in each dual adjacent targets. To make inference tractable, we introduce two robust functions that eliminate the two binary processes. We then propose a novel belief propagation algorithm, called Particle-based belief propagation, and embed it into a Markov chain Monte Carlo approach to obtain the maximum a posteriori (MAP) estimation in the DMN. With a stratified sampler, we incorporate the information obtained from a learned bottom-up detector (e.g., support vector machine (SVM) based classifier) and the motion model of the target into a message propagation framework. Other low-level visual cues, such as motion and shape, can be easily incorporated into our framework to obtain better tracking results. We have performed extensive experimental verification, and results suggest that our method is comparable to the state-of-art multitarget tracking methods in all cases we tested.

This chapter presents an overview of topics and major concepts in unsupervised learning for visual pattern analysis. Cluster analysis and dimensionality are two important topics in unsupervised learning. Clustering relates to the grouping of similar objects in visual perception, while dimensionality reduction is essential for the compact representation of visual patterns. In this chapter, we focus on clustering techniques, offering first a theoretical basis, then a look at some applications in visual pattern analysis. With respect to the former, we introduce both concepts and algorithms. With respect to the latter, we discuss visual perceptual grouping. In particular, the problem of image segmentation is discussed in terms of contour and region grouping. Finally, we present a brief introduction to learning visual pattern representations, which serves as a prelude to the following chapters.

Tracking objects in image sequences involves performing motion analysis at the object level, which is becoming an increasingly important technology in a wide range of computer video applications, including video teleconferencing, security and surveillance, video segmentation, and editing. In this chapter, we focus on sequential Bayesian estimation techniques for visual tracking. We first introduce the sequential Bayesian estimation framework, which acts as the theoretic basis for visual tracking. Then, we present approaches to constructing representation models for specific objects.

Streaming media services have been applied in many applications. At the same time, the security of it should be considered. To protect the video content, watermarking data (image) is usually embedded into the quantized DCT coefficients of video. At the same time, the watermarked video should keep the fidelity and the bit-rate. While for high efficient compression video such as H.264/AVC it is very difficult, because just one bit alteration may widely affect the video content and the bit-rate. A CAVLC-based blind watermarking method for H.264/AVC compressed video is proposed. The watermarking data is only embedded into the last non-zero and non-trailing AC coefficient in context adaptive variable length coding (CAVLC) of H.264/AVC. With this kind of embedding, the artifact due to the embedding could be reduced efficiently by CAVLC. Experimental results show that on average, the introduced distortion by watermark embedding is less than 0.5 dB, and the increased stream bit rate is only 0.1%.

This paper presents two linear regression models by exploring the relationships between DCT coefficients of the original image and filtered image. The first model is to scale the DCT coefficients of the original image in order to approximate the operation of 2-D spatial domain filtering. The second model is to predict the original image from the filtered image in a similar manner. We show that the first model is used for DCT domain filtering, while the second model can be used for fast DCT domain image deblurring. Both of them are easy to implement on compressed formats of DCT-based compression methods (JPEG, MPEG, H.26X) by using decoding quantization tables that are different from the encoding quantization tables.

An efficiently manageable overlay network architecture, called AIRVoD, is proposed for video-on- demand, based on a distributed-centralized P2P network. The system consists of distributed servers, peers, SuperNodes and sub-SuperNodes. The distributed central servers act as the centralized index server to store the peer list, program list, and buffer information of peers. Each newly joined peer in AIRVoD periodically exchanges data availability information with the central server. Some powerful peers are selected to be sub-SuperNodes which store a larger part of the demanded program. The demanding peers can retrieve correspondingly unavailable data from partners selected from the central server, sub- SuperNodes and SuperNodes that have the original programs to supply the available data, by using the proposed parallel scheduling algorithm. There are four characteristics of this architecture: 1) easy to globally balance and manage: central server can identify, and cluster each newly joined peer, and allocate the load in the whole peer network; 2) efficient for dynamic networks: data transmission is dynamically determined according to data availability which can be derived from central server; 3) resilient, as the partnerships can adapt to quick switching among multi-suppliers under global balance; and 4) highly cost-effective: the powerful peers are taken full advantage to be larger suppliers. AIRVoD has been demonstrated based on CERNET OF CHINA.

Combining the advantages of the centralized P2P structure and the data-driven structure with scalable video coding, we propose an adaptive P2P architecture for live video based on scalable wavelet video coding over Internet. The core operations are simple: every peer periodically exchanges data availability and bandwidth information with the central server, acting as the centralized index, which selects and sends a set of partners that have expected data to the demanding node. And the central server classifies one peer to a certain level according to the peer's downloading bandwidth which coordinates with the layer level of the video data encoded using scalable wavelet coding. The peer retrieves correspondingly unavailable data from partners according to availability information. There are three principal advantages of this architecture: 1) easy to manage, as the server authorizes, classifies, and clusters each new peer according to its bandwidth as it joins the overlay network, and thus the servers maintain a global structure; 2) efficiency in dynamically heterogeneous networks, as users with different processing ability under heterogeneous networks can retrieve the adaptive data from their partner peers according to their downloading bandwidths, and 3) robustness and resilience, as the partner peers can adapt to quick switching among multi-suppliers, and the scalable video transmission is adaptive. A scheduling algorithm is proposed to enable efficient and continuous scalable streaming of low to high bandwidth content with different service levels over heterogeneous networks.

Scalable video coding is applied to meet the heterogeneity of networks, fluctuation of bandwidth, and diversity of end users. But if the base layer is damaged or missed, the scalable video coding will no longer be effective. To address this problem, a scalable multiple description of video is proposed based on the bit plane and rate allocation of scalable wavelet video coding in an error-prone transmission environment. After spatiotemporal transformation, three-dimensional bit planes of the wavelet frame are sampled in odd and even order except the most significant bit to generate multiple descriptions, which are encoded into scalable video streaming by using entropy coding and Lagragian rate allocation. Combined with multiple path transport, the performance of proposed scalable multiple description coding is demonstrated in Ns2.

A method of learning image bases from natural images is proposed in this paper. Natural image is decomposed into structure mode and non-structure mode by using Empirical Mode Decomposition (EMD) technique. The structure mode is made up of the first two Intrinsic Mode Functions (IMF) that represent the structure information of image. The other IMFs and residual image compose the non-structure mode. A structure bases dictionary or a non-structure bases dictionary is obtained by using Independent Component Analysis (ICA) on structure mode or non-structure mode respectively. The image can be expressed by using the obtained dictionaries according to different attribution of image regions to implement image coding and sparse representation. Compared with ICA, the structure bases can better capture structure information of natural images, and the separation of the structure bases and non- structure bases can be profitable for the different description of different images information.

This paper presents a manageable overlay network architecture SVCP2P for live scalable media streaming. Every peer in SVCP2P periodically exchanges data availability information with one of distributed central servers which act as the centralized index for storing peer list, program list and buffer information of peers. An efficient scheduling algorithm is proposed, which achieves real-time and continuous transmission of the scalable streaming. There are three characteristics of this architecture: 1) easy management; 2) efficient to heterogeneous network because of the scalable media streaming adapting to the heterogeneous demand; and 3) robust and resilient. We have examined the SVCP2P which has been implemented based on the IP Internet over LAN, and the results demonstrate the efficiency of SVCP2P.

Primal sketch performs an important role in early vision. In this paper, we propose a novel method to obtain the primal sketch of natural images by integrating empirical mode decomposition (EMD) techniques and image gradient. 2D EMD approach can decompose the image into a finite number of intrinsic mode functions (IMF), and each one represents the original image in a different scale, with the 1st IMF representing the finest scale. To enhance the information represented by the IMF, we multiply the 1st IMF by the image gradient. This enhanced IMF highlights intensity changes in the image. By linking all the maximal points in the enhanced IMF, we obtain a primal sketch of the original image. Compared with the existed primal sketch extraction methods, our method is fully driven by the image data, and it needs neither to choose filters nor to learn the image bases. The experiment results show that our method is fast and effective.

To achieve robust system, more and more vision researchers take into account fusing multiple visual cues. In this paper, we propose a novel strategy to integrate multiple naive cues for head tracking. Firstly, a cue dependency model is constructed via graphical model. Secondly, a new inference procedure based on non-parametric belief propagation is built for cue integration. The work presented is thus a general framework easy to extend for other computer vision research problems. Experimental results imply that the strategy we propose is effective, and it is robust without estimation of cue reliability.

This paper presents a robust player tracking method for sports video analysis. In order to track agile player stably and robustly, we employ multiple models method, with a mean shift procedure corresponding to each model for player localization. Furthermore, we define pseudo measurement via fusing the measurements obtained by mean shift procedure. And the fusing coefficients are built from two likelihood functions: one is image based likelihood; the other is motion based association probability. Experimental results show effectiveness of our method in the hard case of player tracking literature.

We propose an image representation, which has the property of level of details. By decomposing the image into level sets and subtracting between two level sets, a level difference set is defined. The connected components of each level difference set are measured using their area. Then the connected components are stored in a data structure in order of size, largest first. The level difference sets-based representation is applied in image-based rendering when time is critical. The data of the image is loaded from the head of the data structure and used to calculate or render. Thus the outlines of the image are displayed first, and details are added as time passes. Some experimental results are demonstrated here to show the efficiency of the method.

In this paper, we propose an approach to constructing reconfigurable vision system. We found that timely and efficient execution of early tasks can significantly enhance the performance of whole computer vision tasks, and abstract out a set of basic, computationally intensive stream operations that may be performed in parallel and embodies them in a series of specific front-end processors. These processors which based on FPGAs (Field programmable gate arrays) can be re-programmable to permit a range of different types of feature maps, such as edge detection & linking, image filtering. Front-end processors and a powerful DSP constitute a computing platform which can perform many CV tasks. Additionally we adopt the focus-of-attention technologies to reduce the I/O and computational demands by performing early vision processing only within a particular region of interest. Then we implement a multi-page, dual-ported image memory interface between the image input and computing platform (including front-end processors, DSP). Early vision features were loaded into banks of dual-ported image memory arrays, which are continually raster scan updated at high speed from the input image or video data stream. Moreover, the computing platform can be complete asynchronous, random access to the image data or any other early vision feature maps through the dual-ported memory banks. In this way, the computing platform resources can be properly allocated to a region of interest and decoupled from the task of dealing with a high speed serial raster scan input. Finally, we choose PCI Bus as the main channel between the PC and computing platform. Consequently, front-end processors' control registers and DSP's program memory were mapped into the PC's memory space, which provides user access to reconfigure the system at any time. We also present test result of a computer vision application based on the system.© (2000) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.