Search

In this paper we present a clustering-based method for representing semantic concepts on multimodal low-level feature spaces and study the evaluation of the goodness of such models with entropy-based methods. As different semantic concepts in video are most accurately represented with different features and modalities, we utilize the relative model-wise confidence values of the feature extraction techniques in weighting them automatically. The method also provides a natural way of measuring the similarity of different concepts in a multimedia lexicon. The experiments of the paper are conducted using the development set of the TRECVID 2005 corpus together with a common annotation for 39 semantic concepts.

The best-basis algorithm has gained much importance on textured-based image compression and denoising of signals. In this paper, an architecture for the wavelet-packet based best-basis algorithm for images is proposed. The paper also describes the architecture for best-tree selection from 2D wavelet packet decomposition. The precision analysis of the proposed architecture is also discussed and the result shows that increase in the precision of input pixel greatly increases the Signal-to-Noise Ratio (SNR) per pixel whereas increase in the precision of filter coefficient does not greatly help in improving the SNR value. The proposed architecture is described in VHDL at the RTL level, simulated successfully for its functional correctness and implemented in an FPGA.

The paper describes a library supporting effective programming and design of matrix algorithms and programs. The important feature of proposed library is the visual algorithm representation using a movie-based approach. The user has a deal with special multimedia objects, each of which can generate an executable code as well as produce animation frames. These objects build an algorithmic skeleton representing the steps of computation. In this paper, we show the main features of movie-based programming as well as describe the movie-based matrix library. Examples of the library usage are also presented.

In this paper, we propose an efficient intra refreshment algorithm, which can achieve both the global optimization in determining the intra refreshing ratio and the local accuracy in selecting the proper macroblocks to be intra coded before the current frame is actually encoded. Furthermore, incorporated with our proposed intra refreshment algorithm, an effective UEP scheme based on a dynamic FMO mapping mode is also proposed in this paper to better protect the intra macroblocks in a frame. The experimental results show that our joint intra refreshment and UEP algorithm can remarkably improve the reconstructed video quality in the packet lossy network

This paper first analyses the relationship between performance and complexity of several state-of-the-art coding algorithms for high resolution videos. Based on the coding efficiency comparison under different config parameters and the intra mode usage in P/B frame, this paper presents a practical scheme to improve the coding speed with slight quality loss. And a DSP-oriented two-level internal memory organization is also proposed to keep pipeline processing. In such organization, block correlation caused by motion vector predictions is lightened while keeping almost the same performance as the original.

This paper describes issues and consideration on authoring of interactive multi-view visual content based on MPEG-4. The issues include types of multi-view visual content; functionalities for user-interaction; scene composition for rendering; and multi-view visual content file format. The MPEG-4 standard, which aims to provide an object based audiovisual coding tool, has been developed to address the emerging needs from communications, interactive broadcasting as well as from mixed service models resulting from technological convergence. Due to the feature of object based coding, the use of MPEG-4 can resolve the format diversity problem of multi-view visual contents while providing high interactivity to users. Throughout this paper, we will present which issues need to be determined and how currently available tools can be effectively utilized for interactive multi-view visual content creation.

XML is a popular approach to interoperable exchange of Multimedia metadata between a wide range of devices. This paper explores extending the use of the Remote XML Exchange Protocol (previously proposed by the authors) as a mechanism to provide efficient interaction with complex Multimedia XML documents and their associated schemas. This is particularly applicable to users with limited application complexity devices and/or limited bandwidth connections. Many XML documents do not fully utilize all the information present in a given schema; thus, users download substantial redundant information for the current application. This paper introduces the use of RXEP for the transmission of small, relevant schema sections. The paper investigates the advantages of schema retrieval using RXEP in terms of the bandwidth saved.

This paper considers optimal cross-layer rate control, scheduling design and power control for multi-hop wireless networks. The problem of optimal rate control, link scheduling, and link transmission power for all active time slot is formulated as a network utility maximization problem. In wireless multi-hop networks, the link capacity is a function of link scheduling and transmission power with time-varying and nonlinear properties. Those characteristic poses much challenge in joint design. To solve the non-convex and non-separable nonlinear program problem, a two time-scale distributed optimization approach is presented. By dual decomposition and gradient method, the NUM problem naturally decomposes into three subproblems: congestion control, scheduling design and power control. They interact through congestion price. The global convergence of this algorithm is proven. This paper presents a step towards a systematic approach to jointly design TCP congestion control algorithms, scheduling design and power control.

In this paper, we propose a remote control system which controls a haptic interface device with another remote haptic interface device. Haptic feeling can be transmitted by the system. Applications of the system are a remote calligraphy system, a remote drawing instruction system, a remote medical operation system, and so on. The paper deals with the remote drawing instruction system by which an instructor trains a learner how to draw pictures or figures while conveying the sense of force through the Internet. In order to clarify what kinds of control are needed in the system, we examine the influences of network delay, delay jitter and packet loss on the output quality of haptic media by subjective assessment of drawing figures.

Multiple descriptions have been widely investigated as a quality of service technology for the delivery of multimedia information over lossy channels. This paper presents generalized methods for transform based multiple descriptions that do not introduce additional distortion to the transformed information. Previous noiseless methods place restrictions on how the information may be quantized prior to correlation. Typically, the information must be quantized with quasi-uniform quantizers which all have the same quantization step size. This paper expands these methods so that the information may be quantized with any quantizer, and different parts of the information may be quantized with different quantization step sizes. For specific correlating transforms, we propose a computationally efficient correlation transform that makes use of the Hadamard matrix

This paper investigates the problem of object surface reflectance modeling, which is sometimes referred to as inverse reflectometry, for photorealistic rendering and effective multimedia applications. A number of methods have been developed for estimating object surface reflectance properties in order to render real objects under arbitrary illumination conditions. However, it is still difficult to densely estimate surface reflectance properties in real-time. This paper describes a new method for real-time estimation of the non-uniform surface reflectance properties in the inverse rendering framework. Experiments are conducted in order to demonstrate the usefulness and the advantage of the proposed methods through comparative study.

Network Channel Coding (NCC) distributes channel coding functions over network nodes participating in common or diverse communication sessions. A particular case of NCC is Network Embedded FEC (NEF) [1]-[2], which has been shown to exhibit significant improvements in the performance of video streaming applications over multicast peer-to-peer (p2p) trees. The placement of NCC/NEF codecs and its utility in improving the throughput performance is in general a function of the underlying p2p graph topology. In this paper we consider two major forms of p2p topologies: (1) perfectly structured k-ary tree topologies that can be built from (virtually) ideal p2p graphs [3] and (2) unstructured random tree topologies where new nodes randomly join as children to any of the existing peers. The two topologies represent an optimal low diameter structured p2p topology and a trivial randomly evolving sub-optimal topology, respectively. In this paper, we show the impact of key graph parameters, such as the maximum node-degree k and minimum tree-height D, on the performance of NCC in terms of NEF throughput as well as video quality for both structured and unstructured topologies. The utility of NCC/NEF for low-degree and/or less structured p2p topologies is especially highlighted by demonstrating that, embedding of additional codecs can render the performance of less structured topologies or higher diameter topologies to be almost as good as that of the very well structured low diameter topologies. We also investigate the impact of the graph properties on the placement of the NEF codecs.

The high definition video adoption has been growing rapidly for the last two years. The two high definition DVD formats HD-DVD and Blueray have mandated MPEG-2, H. 264 and VC-1 as video compression formats. The coexistence of these different video coding standards creates a need for transcoding. In this paper, an efficient transcoding algorithm from VC-1 video to H.264 video is discussed. While there has been recent work on MPEG-2 to H. 264 transcoding, the published work on VC-1 to H.264 transcoding is non-existent. There is very limited amount of published work on VC-1. This paper gives a brief overview of VC-1 and discusses the opportunities for low-complexity tools for VC-1 to H.264 transcoding. The paper considers I and P frame transcoding from VC-1 to H.264 and proposes two approaches 1) Low Cost Design and 2) High Cost Design.

In this paper, we developed a family of 2D and 3D invariant features with applications to 3D human faces recognition. The main contributions of this paper are: (a) systematically deriving a family of novel features, called summation invariant that are invariant to Euclidean transformation in both 2D and 3D; (b) developing an effective method to apply summation invariant to the 3D face recognition problem. Tested with the 3D data from the Face Recognition Grand Challenge v1.0 dataset, the proposed new features exhibit achieves a performance that rivals the best 3D face recognition algorithms reported so far.

In wireless multimedia streaming, there is a need to allow for client prioritisation in order to enable provision of end user perceived quality in direct relationship with client device importance. Currently, the same priority is given to all clients, independent of their characteristics, often resulting in unfair distribution of throughput. This paper proposes a priority-based wireless adaptive multimedia delivery scheme that enables client prioritisation during multimedia distribution over IP networks. The paper presents simulation results outlining the benefits of applying the algorithm, illustrating the improvement in bandwidth allocation and in overall end user perceived quality. The algorithm focuses on a residential wireless local area network and assigns static priorities based on device characteristics.

This paper proposes an effective method for converting any fast DCT algorithm into an approximate multiplierless version. Basically it approximates any constant in the original transform by a signed digit representation. We developed an efficient algorithm to convert any constant into a signed digit string with a minimum number of non-zero signed digits and a reduced length. As the accuracy of an approximated algorithm depends critically on the assignment of signed digits to the constants, this paper formulated an effective algorithm for finding an effective signed digits configuration which could minimize the MSE of an approximated DCT algorithm with a specified complexity. Experiment results show that the AAN's fast DCT algorithm, approximated by the proposed method and using an optimized configuration can be used to reconstruct images with high visual quality in terms of PSNR.

Due to the high performance of recent computer graphics hardware, 3D CG and CG animations have become in great demand for various applications. Many 3D models have already been created and stored. We need any 3D model search system that allows us to retrieve our required 3D models accurately. In this paper, the authors propose a 3D model search system that uses the attributed relational graph (ARG) of each 3D model, and that employs the stochastic ARG matching to measure the similarity among them. The proposed system gives better search results than those of D2 method in their experiments. This paper also describes what kinds of features of a 3D model are used as each vertex attributes for the stochastic ARG matching.

In this paper we propose a system design for delivery of immersive communications to mobile wireless devices based on a distributed proxy model. It is demonstrated that this architecture addresses key technical challenges for the delivery of these services, that is, constraints on link capacity and power consumption in mobile devices. However, additional complexity is introduced with respect to mobility management. The paper proposes three possible methods for updating proxy assignments in response to mobility and compares the performance of these methods.

In this paper, we discuss the cache replacement policy in a multimedia transcoding proxy. Unlike the cache replacement for conventional web objects, to replace some elements with others in the cache of a transcoding proxy, we should further consider the inter consider the inter-relationship among the cached items. To maintain the inter-relationship and to perform cache replacement, we propose in this paper the RESP framework (standing for Replacement with Shortest Path). The RESP framework contains two primary components, i.e., procedure MASP (standing for Minimum Aggregate Cost with Shortest Path) and algorithm EBR (standing for Exchange-Based Replacement). Procedure MASP maintains the inter-relationship using a shortest path table, whereas algorithm EBR performs cache replacement according to an exchanging strategy. The experimental results show that the RESP framework can approximate the optimal cache replacement with much lower execution time for processing user queries.

Highlight summarization technology has been studied widely in sports video analysis. In this paper, we propose a highlight summarization system based on replays. First the replay clips in the sports video are extracted as the highlight candidates. Then the features including audio energy and motion activity are employed to rank the arousal level of the replay clips. Finally we model the highlight with the arousal rank to generate summarization. The contribution of this paper concentrates on two aspects. Firstly, Event-Replay (ER) structure is proposed and some new features are employed to represent the arousal levels of ER for general sports video. Secondly a novel highlight model is proposed considering the inter-relation of ERs. The experiments evaluate the rationality of the system.

A new edge-sensing demosaicking solution based on a novel vector spectral model is introduced in this paper. Using the vector spectral model, the proposed solution preserves the magnitude and the directional characteristics of the single-sensor captured image in both smooth and edge areas. Experimentation reported in this paper indicates that the proposed demosaicking solution produces visually pleasing color images and can outperform the powerful demosaicking schemes in terms of the performance.

Pattern mining algorithms are often much easier applied than quantitatively assessed. In this paper we address the pattern evaluation problem by looking at both the capability of models and the difficulty of target concepts. We use four different data mining models: frequent itemset mining, k-means clustering, hidden Markov model, and hierarchical hidden Markov model to mine 39 concept streams from the a 137-video broadcast news collection from TRECVID-2005. We hypothesize that the discovered patterns can reveal semantics beyond the input space, and thus evaluate the patterns against a much larger concept space containing 192 concepts defined by LSCOM. Results show that HHMM has the best average prediction among all models, however different models seem to excel in different concepts depending on the concept prior and the ontological relationship. Results also show that the majority of the target concepts are better predicted with temporal or combination hypotheses, and there are novel concepts found that are not part of the original lexicon. This paper presents the first effort on temporal pattern mining in the large concept space. There are many promising directions to use concept mining to help construct better concept detectors or to guide the design of multimedia ontology.

In this paper we present a novel method on how to take advantage of the snake representation of target objects, when doing chamfer matching for detection/recognition purposes. In this case several time-consuming steps of classic chamfer matching approaches can be simplified. Moreover, we investigate the possibility of involving fewer pixels from both the target and template object to speed up computations. We introduce an optimization method for such an object reduction, which is valid also in the general application scheme of chamfer matching. Finally, we present our experimental results regarding human body detection

Realizations of demanding embedded applications particularly in the field of signal processing often require high processing performance and low energy consumption which are far beyond what can be delivered by DSPs nowadays. A novel data-parallel coprocessor basing on the transport triggered architecture (TTA) is presented in this paper. The coprocessor consists of two powerful arithmetic clusters, the stream memory as well as the optimized data transport network, and is good at exploiting the data parallelism in the computation intensive multimedia applications. Also, the solution of the clock-gating in a gradual way is advanced for the low power dissipation. Then, a SoC chip involving the coprocessor is implemented using 0.18 mum CMOS process. Experimental results show that this coprocessor has good performance improvement for multimedia applications

This paper proposes a music signal synthesis scheme that is based on sinusoid modeling and sliding-window ESPRIT. Despite widely used audio coding standards, effectively synthesizing music using sinusoid models, more suitable for harmonic rich music signals, remains an open issue. In the proposed scheme, music signals are modeled by a sum of damped sinusoids in noise. A sliding window ESPRIT algorithm is applied. A continuity constraint is then imposed for tracking the time trajectories of sinusoids in music and for removing spurious spectral peaks in order to adapt to the changing number of sinusoid contents in dynamic music. Simulations have been performed to several music signals with a range of complexities, including music recorded from banjo, flute and music with mixed instruments. The results from listening and spectrograms have strongly indicated that the proposed method is very robust for music synthesis with good quality.

The MPEG-4 Scalable to Lossless (SLS) audio coding is recently being developed to provide a unified solution for high - compression perceptual audio coding and high-quality lossless audio coding. SLS provides efficient Fine Granular Scalable (FGS) coding from AAC core layer to lossless, and achieves reasonable perceptual quality at its scalable coding range using a sequential bit-plane scanning method, which minimizes the audio distortion according to the spectral shape of the core layer quantization errors. In this paper, it is shown that the perceptual quality performanc of SLS at intermediate rates can be further improved by incorporating psychoacoustic model into the bit-plane coding process. In addition, it is also found that such an improvement can be achieved by slightly tweaking the original bit-plane coding process of SLS and hence preserving its nice features such as compatibility to lossless coding and low complexity.

Content-based image retrieval (CBIR) systems have been actively investigated over the past decade. Several existing CBIR prototypes claim to be designed based on perceptual characteristics of the human visual system, but even those who do are far from recognizing that they could benefit further by incorporating ongoing research in vision science. This paper explores the inclusion of human visual perception knowledge into the design and implementation of CBIR systems. Particularly, it addresses the latest developments in computational modeling of human visual attention. This fresh way of revisiting concepts in CBIR based on the latest findings and open questions in vision science research has the potential to overcome some of the challenges faced by CBIR systems.

In general, digital images can be classified into photographs and computer graphics. This taxonomy is very useful in many applications, such as web image search. However, there are no effective methods to perform this classification automatically. In this paper, we manage to solve this problem from two aspects. At first, we propose some novel low-level features that can reveal perceptional differences between photographs and graphics. Then, we adopt an effective algorithm to perform the classification. The experiments conducted on a large-scale image database indicate the effectiveness of our algorithm.

The line-based face recognition method is distinguished by its features, but its development and application is limited to some inherent drawbacks. This paper propose a method for decreasing the influence under variable illumination intensity by using the line-based singular value (LSV) feature vector instead of image gray-level value to calculate "distance" between two lines. We prove that our method is invariant to the illumination intensity. Finally, we suggest a distributed computing algorithm using grid computing to solve the multi-scale computation. Experimental results show our approach is effective.

Multi-reference frame motion estimation improves the accuracy of motion compensation in video compression, but it also dramatically increases computational complexity. Based on tracing motion vector trajectories, fast approximated motion estimation results can be obtained for multi-reference frames. In this paper, we extend the downhill simplex search to multiple reference frames and propose several enhanced schemes to improve its efficiency and accuracy. Experimental results show that the proposed algorithm outperforms several representative single-reference frame block matching methods.

In this paper, we propose an improved resource reservation algorithm ESRPT (Enhanced Shortest Remaining Processing Time) for bursty traffic based on IEEE 802.15.3 standard. In this algorithm, each transmitter reports current fragments number of the first MSDU (MAC Service Data Unit) and all of the fragments number of the remainder MSDUs in the pending transmission queue to PNC (piconet coordinator). In the next superframe, PNC firstly allocates part CTAs (Channel Time Allocation) for each stream based on the remainder fragments number of the first MSDU by SRPT rule, then allocates remainder CTAs for each stream based on all fragments number of remainder MSDUs by the same SRPT rule. This algorithm decreases the job failure rates of delay-sensitive video streaming efficiently at the cost of slightly increasing communication overheads. Simulation results show that our proposed ESRPT method achieves better performance in QoS for multimedia streams compared to the existing SRPT schemes.

In this paper, we propose a novel hybrid de-interlacing algorithm, which effectively combines two motion-compensated (MC) de-interlacing techniques: MC Median Filtering (MCMF) and Adaptive Recursive (AR) with one spatial approach: Line Averaging (LA). Despite of its drawbacks, AR is one of the best methods nowadays. MCMF helps reduce flickers and LA is very robust to erroneous motion vectors. The interpolation switches among these methods based on the proposed measurement of texture smoothness and motion vector (MV) reliability. MCMF is adopted when MV is reliable and texture is rich. LA is used when MV is unreliable and texture is smooth. AR is applied to the remaining regions. Experimental results show that the proposed algorithm is superior to the compared algorithms in terms of peak signal-to-noise ratio (PSNR) and that the de-interlaced videos have very high subjective quality.

In this paper, we propose an efficient algorithm, called MQS-change hnge (changes of Music Query Streams), to detect the changes of maximal melody structures in user-centered music query streams. Two music melody structures (set of chord-sets and string of chord-sets) are maintained and four melody structure changes (positive burst, negative burst, increasing change and decreasing change) are monitored in a new data structure MSC-list (a list of Music Structure Changes). Experiments show that MQS-change algorithm is an online, single-pass approach to detect the changes of music melody structures over continuous music query streams.

Numerous approaches can be employed in exploiting computation power in processors such as superscalar, VLIW, SMT and multi-core on chip. In this paper, a UniCore VisoMT processor is proposed, which unifies VLIW and multithreading by providing an efficient control and data communication model, while offering explicit parallelisms for embedded applications. The architecture concurrently executes a main thread and several accelerative threads, coordinated by the main thread. A switch-based register-file is provided for fast data exchange between these accelerative threads. Moreover, a SMT helper function unit is employed for controlling and resource-sharing between accelerative threads, and an event-driven mechanism is introduced for synchronization between the main thread and these accelerative threads. Our results show that the proposed architecture provides area and performance advantages for embedded multimedia applications.

Automatic facial expression analysis is an important aspect of human machine interaction as the face is an important communicative medium. We use our face to signal interest, disagreement, intentions or mood through subtle facial motions and expressions. Work on automatic facial expression analysis can roughly be divided into the recognition of prototypic facial expressions such as the six basic emotional states and the recognition of atomic facial muscle actions (action units, AUs). Detection of AUs rather than emotions makes facial expression detection independent of culture-dependent interpretation, reduces the dimensionality of the problem and reduces the amount of training data required. Classic psychological studies suggest that humans consciously map AUs onto the basic emotion categories using a finite number of rules. On the other hand, recent studies suggest that humans recognize emotions unconsciously with a process that is perhaps best modeled by artificial neural networks (ANNs). This paper investigates these two claims. A comparison is made between detection of emotions directly from features vs. a two-step approach where we first detect AUs and use the AUs as input to either a rulebase or an ANN to recognize emotions. The results suggest that the two-step approach is possible with a small loss of accuracy and that biologically inspired classification techniques outperform those that approach the classification problem from a logical perspective, suggesting that biologically inspired classifiers are more suitable for computer-based analysis of facial behavior than logic inspired methods

This paper presents a new framework that integrates relevance feedback into region-based image retrieval (RBIR) systems based on radial basis function network (RBFN). A modified unsupervised subtractive clustering algorithm is proposed for RBFN center selection according to the characteristics of region-based image representation. A new kernel function of RBFN is introduced for image similarity comparison under region-based representation. The underlying network parameters (weight and width) are then optimized using a supervised gradient-descent training strategy. Experimental results using a database of 10,000 images demonstrate the effectiveness of the proposed hybrid learning approach.

Multimodal surveillance systems using visible/IR cameras and other sensors are widely deployed today for security purpose, particularly when subjects are at a large distance. However, audio information as an important data source has not been well explored. One of the reasons is because audio detection using microphones needs installation close to the subjects in monitoring. In this paper, we investigate a novel "optical" sensor, called Laser Doppler Vibrometer (LDV), for capturing voice signals in a very large range to realize a truly remote and multimodal surveillance system. Speech enhancement approaches are studied based on the characteristics of LDV Audio. Experimental results show that remote voice detection via an LDV is promising when choosing appropriate targets close to human subjects in the environment.

The demand for automatically annotating and retrieving medical images is growing faster than ever. In this paper, we present a novel medical image retrieval method based on SEMI-supervised Semantic Error-Correcting output Codes (SEMI-SEC). The experimental results on IMAGECLEF 2005 [1] annotation data set clearly show the strength and the promise of the presented methods.

In future video communication services, the user's communication device, such as PC, laptop, PDA or mobile phone is equipped with new interaction modalities. These can be cameras and microphones on the capturing side and speech synthesis and video/3D graphics on the rendering side. Haptic and tactile interfaces become also available. These modalities help the user to interact more intuitive with complex devices and tools and provide new services. Hence, a key challenge of new modalities is robustness and stability under general conditions in arbitrary environments. Furthermore, inexperienced users should be able to use these new capabilities without dedicated knowledge of device settings or algorithms. In this paper, we will present some key components for a robust vision-based user interface, which are integrated in an advanced future video communication service

Semantic video content extraction and selection are critical steps in sports video analysis and editing. The identification of video segments can be from various semantic perspec-tives, e.g. certain event, player or emotional state. In this paper, we examined the possibility of automatically identifying shots with "happy" or "sad" emotion from broadcast sports video. Our proposed model first performs the sports highlight extraction to obtain candidate shots that possibly contain emotion information and then classifies these shots into either "happy" or "sad" emotion groups using Hidden Markov Model based method. The final experimental results are satisfactory.

In order to enable transparent and convenient use of multimedia content across a wide range of networks and devices, content adaptation is an important issue within multimedia frameworks. The so called Digital Item Adaptation (DIA) standard is one of the core concepts of the MPEG-21 framework that will support the adaptation of multimedia resources according to device capabilities, underlying network characteristics, and user preferences. Most multimedia adaptation engines for providing Universal Multimedia Access (UMA) scale the content with respect to terminal capabilities and resource constraints. This paper focuses on the cross-modal adaptation decision taking process considering the user environment and terminal capabilities as well as resource limitations on the server, network, and client side. This approach represents a step toward increased Universal Multimedia Experience (UME). Based on four different algorithms for solving this optimization process, we present an evaluation of results gained by running their implementations on different test networks.

The use of block transforms for coding intra-frames in video coding may preclude higher coding performance due to residual correlation across block boundaries and insufficient energy compaction, which translates into unrealized rate-distortion gains. Subjectively, the occurrence of blocking artifacts is common. Post-filters and lapped transforms offer good solutions to these problems. Lapped transforms offer a more general framework which can incorporate coordinated pre- and post-filtering operations. Most common are fixed lapped transforms (such as lapped orthogonal transforms), and also transforms with adaptive basis function length. In contrast, in this paper we determine a lapped transform that non-linearly adapts its basis functions to local image statistics and the quantization regime. This transform was incorporated into the H.264/AVC codec, and its performance evaluated. As a result, significant rate-distortion gains of up to 0.45 dB (average 0.35dB) PSNR were obtained compared to the H.264/AVC codec alone

In a typical image retrieval system, all visual features of query images are used to determine image similarity. Thus, users are left to decide whether or not to include images that not only contain desirable features but also irrelevant ones. Fewer examples or a contaminated set of more could compromise the retrieval effectiveness of most similarity measures. In this paper, we extend our previous approach that allows users define queries by specifying relevant features present in image examples. The extended technique support queries decomposed in multiple clusters, each forming a subquery. Our experimental results have shown a remarkable improvement in retrieval performance

Global rule induction technique has been successfully used in information extraction (IE) from text documents. In this paper, we employ the technique to identify story boundaries in news video. We divide our framework into two levels: shot and story levels. We use a hybrid algorithm to classify each input video shot into one of the predefined genre types and employ the global rule induction technique to extract story boundaries from the sequence of classified shots. We evaluate our rule induction based system on ~120-hours of news video provided by TRECVID 2003. The results show that we could achieve an F 1 accuracy of over 75%

Mining multimedia data is one of the most important issues in data mining. In this paper, we propose an online one-pass algorithm to mine the set of frequent temporal patterns in online music query streams with a sliding window. An effective bit-sequence representation is used to reduce the processing time and memory needed to slide the windows. Experiments show that the proposed algorithm only needs a half of memory requirement of original music query data, and just scans the data once.

Bit rate adaptation is one of the most important types of video transcoding. With H. 264 becoming the predominant video codec of choice in video coding and streaming, prudent rate-adaptation techniques should be developed. In this paper, we investigate certain critical points in the spectrum of rate shaping requests. We show that the selection of quantization step sizes may not have monotonic effects on rate-distortion characteristics in the transcoding sense. In other words, distortion in the regular sense can be different from the distortion in the context of transcoding in which a requantization process is carried out. We show in a generic form that careful selections of the step size can lead to much improved performance. Experiments based on both simulation and real transcoding show the effectiveness of the proposed solution.

This paper presents an emotion recognition system from clean and noisy speech. Geodesic distance was adopted to preserve the intrinsic geometry of emotional speech. Based on the geodesic distance estimation, an enhanced Lipschitz embedding was developed to embed the 64-dimensional acoustic features into a six-dimensional space. In order to avoid the problems brought by noise reduction, emotion recognition from noisy speech was performed directly. Linear Discriminant Analysis (LDA), Principal Component Analysis (PCA) and feature selection by Sequential Forward Selection (SFS) with Support Vector Machine (SVM) were also included to compress acoustic features before classifying the emotional states of clean and noisy speech. Experimental results demonstrate that compared with other methods, the proposed system makes approximately 10% improvement. The performance of our system is also robust when speech data is corrupted by increasing noise.

Video Shot boundary detection is a fundamental task in any kind of video content manipulation and retrieval. The area of shot boundary detection has been extensively studied, but achieving highly accurate detection results remains a challenge. In this paper, we present a novel algorithm for video cut detection. The algorithm is implemented and evaluated on the TRECVID benchmark platform. The experimental results show the effectiveness of the proposed approach. The algorithm provides a clue to the gradual transition detection.

In this paper, we propose an original framework for three dimensional face representation and matching for identification purposes. Basic traits of a face are encoded by extracting curves of salient ridges and ravines from the surface of a dense mesh. A compact graph representation is then extracted from these curves through an original modeling technique capable to quantitatively measure spatial relationships between curves in a three dimensional space. In this way, face recognition is obtained by matching 3D graph representations of faces. Experimental results on a 3D face database show that the proposed solution attains high recognition accuracy and is quite robust to facial expression and pose changes.

This paper describes a rate control algorithm for a streaming video system that dynamically transcodes stored JPEG 2000 frames. The proposed algorithm is designed to improve overall quality over a static rate control method by increasing bandwidth utilization, while satisfying buffer constraints and maintaining consistent quality over time. Simulation results confirm the effectiveness of the proposed algorithm in terms of both objective measures and subjective evaluation.