Search

This paper focuses on the comparison between two fusion methods, namely early fusion and late fusion. The former fusion is carried out at kernel level, also known as multiple kernel learning, and in the latter, the modalities are fused through logistic regression at classifier score level. Two kinds of multilayer fusion structures, differing in the quantities of feature/kernel groups in a lower fusion layer, are constructed for early and late fusion systems, respectively. The goal of these fusion methods is to put each of various features into effect and mine redundant information of the combination of them, and then to develop a generic and robust semantic indexing system to bridge semantic gap between human concepts and these low-level visual features. Performance evaluated on both TRECVID2009 and TRECVID2010 datasets demonstrates that the systems with our proposed multilayer fusion methods at kernel level perform more stably to reach the goal than the classification-score-level fusion; the most effective and robust one with highest MAP score is constructed by early fusion with two-layer equally weighted composite kernel learning.

Prosodic structure generation is the key component in improving the intelligibility and naturalness of synthetic speech for a text-to-speech (TTS) system. This paper investigates the problem of automatic segmentation of prosodic word and prosodic phrase, which are two fundamental layers in the hierarchical prosodic structure of Mandarin, and presents a two-stage prosodic structure generation strategy. Conditional random fields (CRF) models are built for both prosodic word and prosodic phrase prediction at the front end with different feature selections. Besides, a transformation-based error-driven learning (TBL) modification module is introduced in the back end to amend the initial prediction. Experiment results show that the approach combining CRF and TBL achieves an F-score of 94.66 %.

Three factors, including churn of peers, high transmission delay, and high bandwidth heterogeneity, jointly bring forward great challenges to video streaming over P2P networks. In this paper, the multi-tree approach is leveraged to construct an overlay with resilience to churn and low transmission delay. For such a multi-tree structured overlay, a server-aided adaptive video streaming scheme is proposed to cope with the bandwidth heterogeneity. During streaming process, video data are collaboratively forwarded to the same receiver by multiple peers based on side information and network condition, as well as the distributed bitstream is dynamically switched among multiple available versions in a rate-distortion optimized way by the streaming server. Simulation results show that the proposed scheme achieves great gain in overall perceived quality over simple heuristic schemes.

Advanced video coding is recently announced and widely used, although the according protection means have not been developed thoroughly. In this paper, a secure AVC coding scheme is presented based on some partial encryption algorithms. During AVC encoding, such sensitive data as intra-prediction mode, residue data and motion vector are encrypted partially. Among them, the infra-prediction mode is encrypted based on exp-Golomb entropy coding, the intra-macroblocks' DCs are encrypted based on context-based adaptive variable length coding (CAVLC), and infra-macroblocks' ACs and the inter-macroblocks' MVDs are sign-encrypted with a stream cipher followed with variable-length coding. This encryption scheme is secure in perception, keeps format compliance, and obtains high time efficiency though reducing the encrypted data volumes. These properties make it practical to incorporate encryption/decryption process into compression/decompression process, and thus suitable for secure video transmission or sharing.

Some joint fingerprinting and decryption schemes were reported recently for secure multimedia distribution. However, most of them need to be investigated before practical applications. In this paper, the secure distribution scheme proposed by Lemma et al. is investigated and improved. Since this scheme aims to distribute multimedia content by encryption and watermarking, some important performances determine its practicability, including the perceptual security of the encryption operation, the imperceptibility of the embedded watermark and the robustness of the embedded watermark. Some flaws are found in the scheme, such as the low encryption strength, the data overflow caused by encryption/decryption and the low correlation value caused by collusion, which degrade its performances greatly. To improve the scheme, some means are proposed, including media preprocessing, media encryption based on module addition and collusion-resistant fingerprint encoding. Comparative experiments show that better performances are obtained by the improved means. The analysis method proposed in this paper can be used to investigate some other joint fingerprinting and decryption schemes.

In sports programs, there are many special cases making shot boundary detection difficult. Targeted for these special cases, not be considered by existing work, this paper presents a shot boundary detection scheme to detect both cuts and gradual transition efficiently. For shot detection, the algorithm is proposed to resist continuous flashes, camera occlusion or image blur that have not been considered before. For gradual transition detection, a unified method is presented to detect various transitions or special effects, together with an algorithm to reduce the false positives caused by fast camera or object motions. The cut detection and gradual transition detection are implemented serially to avoid repeated detection operations. Compared with existing typical works, the proposed scheme obtains higher correct detection rate and fast detection speed, and is more suitable for sports program analysis.

Ubiquitous multimedia applications are becoming more and more popular. However, the solutions that confirm content and interaction security in these applications are still open issues because of various network convergences and device interconnections. This paper investigates a ubiquitous multimedia scheme and proposes a secure service solution. In the ubiquitous scheme, the multimedia content is encoded with scalable video coding and broadcasted via Digital Video Broadcasting for Handheld Terminals (DVB-H) to mobile terminals, the access right is transmitted by Global System for Mobile (GSM) channel, and the media content and access right can also be transmitted from mobile terminals to home TV through WiFi based Wireless Local Area Network. The proposed secure solution supports three kinds of business models by using various content encryption modes and secure interaction protocols. The solution's security is evaluated and discussed. Since few works have been done to solve this problem, this paper is expected to attract more researchers.

Joint factor analysis (JFA) has been successfully applied to speaker verification tasks to tackle speaker and session variability. In the sense of Bayesian statistics, it is beneficial to take account of the uncertainties in JFA to better characterize its speaker enrollment and verification processes, e.g. representing target speaker model by posteriori distribution of latent speaker factors and evaluating model likelihood by integrating over all latent factors. However, in a JFA model which has a large number of latent factors, it is computationally demanding to carry out these things in their exact form. In this paper, an alternative approach based on variational Bayesian is developed to explore uncertainties in JFA in an approximate yet efficient way. In this method, fully correlated posteriori distribution is approximated by a variational distribution of factorial form to facilitate inference; and a tight lower bound on model likelihood is derived. Experimental results on the 10sec4w-10sec4w task of the 2006 NIST SRE show that variational Bayesian JFA could obtain better performance than JFA using point estimate.

From the results of the NIST speaker recognition evaluation in resent years, speaker recognition systems which are mainly developed based on English training data suffer the language gap problem, namely, the performance of non-English trails is much worse than that of English trails. This problem is addressed in this paper. Based on the conventional joint factor analysis model, we enrolled in the language factors which are mean to capture the language character of each testing and training speech utterance, and compensation was carried out by removing the language factors in order to shrink the difference between languages. Experiments on 2006 NIST SRE data show that, the language factor compensation alone can reduce the gap between the performance of English and non-English trails, and the score level combination with eigenchannels can further improve the performance of non-English trails, e.g., for female part, we observed about 19% relatively reduction in EER, when compared with eigenchannels session variability compensation alone.

Text normalization is an important component in mandarin Text-to-Speech system. This paper develops a taxonomy of Non-Standard Words (NSW's) based on a Large-scale Chinese corpus and proposes a three-stage text normalization strategy: Finite State Automata (FSA) for initial classification, Maximum Entropy (ME) Classifier & Rules for further classification and General Rules for standard word conversion. The three-stage approach achieves Precision of 96.02% in experiments, 5.21% higher than that of simple rule based approach and 2.21% higher than that of simple machine learning method. Experiments results show that the approach of three-stage disambiguation strategy for text normalization makes considerable improvement, and works well in real TTS system.

Cross-document coreference resolution, which is an important subtask in natural language processing systems, focus on the problem of determining if two mentions from different documents refer to the same entity in the world. In this paper we present a two-step approach, employing a classification and clusterization phase. In a novel way, the clusterization is produced as a graph cutting algorithm, namely, neural networks-based BestCut (NBCut). To our knowledge, our system is the first that employs a statistical model in graph partitioning. We evaluate our approach on ACE 2008 cross-document coreference resolution data sets and obtain encouraging result, indicating that on named noun phrase coreference task, the approach holds promise and achieves competitive performance.

For concatenative speech synthesis based on non-uniform unit selection, the key to improve the synthetic quality is the careful designing of measuring criteria respect to the units adopted. With our previous hierarchical non-uniform unit selection framework (Xu et al., 2007), two measurements for selecting optimal non-uniform units during searching at different layers are proposed in this paper, including inter-syllable pitch control and spectra distance by phonetic context. These measures are used as components of our cost function, especially for boundaries in front of syllables starting with voiceless consonants. Experiment shows it outperforms our previous system.

Some joint fingerprinting and decryption (JFD) schemes are recently reported. However, most of them need to be investigated before practical applications. In this paper, the audio JFD scheme proposed by Lemma et al. is investigated and evaluated. Some flaws are found in the scheme, such as the low encryption strength, the data overflow caused by encryption and the fragileness to collusion attacks. To improve the scheme, some means are proposed, such as the audio encryption without overflow and the collusion-resistant fingerprint encoding. Comparative experiments show that better performances are obtained by the improved means. The analysis method proposed in this paper can be used to investigate some other JFD schemes.

Text normalization is an important component in text-to-speech system and the difficulty in text normalization is to disambiguate the non-standard words (NSWs). This paper develops a taxonomy of NSWs on the basis of a large scale Chinese corpus, and proposes a two-stage NSWs disambiguation strategy, finite state automata (FSA) for initial classification and maximum entropy (ME) classifiers for subclass disambiguation. Based on the above NSWs taxonomy, the two-stage approach achieves an F-score of 98.53% in open test, 5.23% higher than that of FSA based approach. Experiments show that the NSWs taxonomy ensures FSA a high baseline performance and ME classifiers make considerable improvement, and the two-stage approach adapts well to new domains.

The phonetic decision tree-based state tying is a widely adopted way to establish a high accuracy CD (context- dependent) speech recognizer with HMM (hidden Markov model) approach. To obtain a set of reasonable phonetic questions, great contributions from phonetic experts with the knowledge of certain language are expected. It is a time consuming and expensive process which unfortunately happens every time a new phoneme set is defined. This paper introduces an efficient process to automatically generate phonetic question sets by which a CD speech recognizer can achieve same level of performance in comparison to the case of phonetic question set from experts' knowledge.

Many data aggregation schemes based on kinds of clusters have been proposed for reducing energy cost in WSN by leveraging the correlation or redundancy among sensor nodes. But most of the clustering protocols are based on k-hops neighbors in spite of the data distribution over the neighbor nodes. When cluster head is selected as the representative node for all the member nodes to report aggregated data to sink, there may be significant errors due to the fluctuating data distribution within the cluster. In this paper, we propose a data-aggregation with energy and data aware cluster (DEDAC) in which the clusters are formed according to the data values of the sensor nodes instead of the neighbor relation. Simulation results show that our DEDAC can achieve more accurate aggregation data while retaining the feature of energy efficient.

In speech synthesis systems based on wave concatenation, using longer units can generate more natural synthetic speech. In order to improve the usage of longer units in the corpus, this paper proposed a hierarchical non-uniform unit selection framework. Each layer included in the framework is an independent searching procedure which searches for different sized units and adopts suitable naturalness measuring functions related to the unit type. We have applied it to our Mandarin speech synthesis system according to the Chinese prosodic structure with respect to the statistical result in our corpus. Experiment result shows it outperforms our previous system.

This paper comparatively evaluated various knowledge sources and smoothing algorithms for pronunciation disambiguation in Mandarin TTS (text-to-speech) systems under maximum entropy (maxent) framework. In particular, five kinds of knowledge sources, namely characters and their pronunciations, words, their pronunciations and part-of-speech. together with two smoothing algorithms, i.e. Gaussian prior and inequality were compared. In our experiments conducted on 107 key Chinese polyphones. we found that all the knowledge sources almost perform equally well given the same smoothing measure, but the character-based features compare favorably because they are language independent and can be obtained with the lowest computation cost. Compared with the widely-used Gaussian smoothing, the equality smoothing greatly reduces the number of active features and yields a slightly improved accuracy on each knowledge source. Our best result (96.36%) is achieved by using character-based features together with the inequality smoothing, significantly superior to 81.22% by selecting the most frequent pronunciations and 88.72% by dictionary look-up with the part-of-speech. We also compared the maxent classifier with the transform-based error-driven learning algorithm (E. Brill, 1995) using the same knowledge sources, the results show that the maxent classifier achieve better performance to solve the polyphone disambiguation.

In speaker recognition fields, score normalization is a widely used and effective technique to enhance the recognition performances and is developing further. In this paper, we are focused on the comparison among many kinds of candidates of score normalization methods and a new implementation of the speaker adaptive test normalization (ATnorm) based on a cross similarity measurement is presented which doesn't need an extra corpus for speaker adaptive impostor cohort selection. The use of ATnorm for the language robustness of the multi-language speaker verification is also investigated. Experiments are conducted on the core task of the 2006 NIST Speaker Recognition Evaluation (SRE) corpus. The experimental results indicate that all the score normalization methods mentioned can improve the recognition performances and ATnorm behaves best. Moreover, ATnorm can further contribute to the performance as a means of language robustness.

The performance of a TTS system relies on the acoustic features of output speech, while we have only the linguistic information in text context for the corpus design phase. To reconcile that conflict, this paper proposes a design method based on decision tree. First, trees are trained using an existing speech corpus: the splitting questions are selected from context vectors and the distance metric is based on acoustic features. Then units from a new text corpus which will be the source of target script are inserted into the trees. The sentence selection strategy is a combination of coverage of major tree nodes and frequent context vector clusters. Experiment is carried out by collecting a Beijing Olympics related corpus taking advantage of an existing domain-unspecified speech corpus and supplementary domain-specified text. Informal listening test on TTS outputs confirms that the proposed method achieves a better optimization of speech. It includes not only phonetic balance as conventional methods do, but prosodic balance as well.

Grapheme-to-phoneme (G2P) conversion is an important component in TTS systems. The difficulty in Chinese G2P conversion is to disambiguate the polyphones. In this paper, we formulate the polyphone disambiguation problem into a classification problem and propose a language independent classifier based on maximum entropy to address the issue. Furthermore, we introduce inequality smoothing to alleviate data sparseness and exploit language independent character features as linguistic knowledge. Experimental results show that the character features perform as well as the language dependent features such as words and part-of-speech, compared with the widely-used Gaussian smoothing, the inequality smoothing can greatly reduce the active features used in the classifier and achieve better performance. Our classifier achieves 96.35% in term of overall accuracy, greatly superior to 81.22% by using high-frequent "pin-yin"(Romanization of Chinese phoneme). Finally, we explore to merge all key polyphones into 6 groups and find that the overall accuracy only decreases about 2% and the active features are reduced more than 33% further.

In this paper, a secure multimedia browsing scheme is proposed, which is constructed based on perceptual multimedia encryption and secure key distribution. In this scheme, multimedia data are encrypted perceptually under the control of user key and quality factor. This encryption process combining with Advanced Video Coding (AVC) is of low cost, and keeps file format unchanged. The key distribution scheme deals with user input, authenticates users, and controls the secure multimedia sharing process. Thus, only the users who have registered can obtain multimedia data. And they can be classified into several types according to their payment. The analyses and experimental results show that it is suitable for secure multimedia applications such as Video-on-Demand (VOD) system, Audio-on-Demand (AOD) system, pay-TV, videoconferencing systems, wireless or mobile multimedia, and so on.

A secure image transmission scheme based on JPEG2000 codec is proposed in this paper, which combines encryption with encoding and is suitable for real-time applications. In this scheme, the sensitive data are self-authenticated, then partial-encrypted during compression process, and the compressed data are encrypted by the lightweight encryption algorithm combined with error-correction codec. The self-authentication process can detect malicious tampering or transmission errors. The encryption process obtains suitable tradeoff between security and time-efficiency through encrypting data adaptively, and keeps the original system's error-robustness. The decryption process is symmetric to the encryption process. Experimental results show that this scheme obtains high perception security and time efficiency, and is thus suitable for secure image transmission over network.

Utterance verification is a process, in which a spoken utterance is verified against the given keyword. This process is used to make a decision on acceptance or rejection. In this paper, we propose a new approach to the utterance verification, using a boosting classifier with ten confidence measures. This classifier combines a set of 'weak' learners into a 'strong' one. The experimental results present that it can remarkably improve the verification performance. Compared with a single confidence measure, the equal error rate is reduced by up to 23%. The results also show that the boosting classifier is better than the SVM and MLP classifiers, in term of the equal error rate.