Search

Varying types of shots is a fundamental element in the language of film, commonly used by a visual storytelling director. The technique is often used in creating professional recordings of a live concert, but meanwhile may not be appropriately applied in audience recordings of the same event. Such variations could cause the task of classifying shots in concert videos, professional or amateur, very challenging. To achieve more reliable shot classification, we propose a novel probabilistic-based approach, named as coherent classification net (CC-Net), by addressing three crucial issues. First, we focus on learning more effective features by fusing the layer-wise outputs extracted from a deep convolutional neural network (CNN), pretrained on a large-scale data set for object recognition. Second, we introduce a frame-wise classification scheme, the error weighted deep cross-correlation model (EW-Deep-CCM), to boost the classification accuracy. Specifically, the deep neural network-based cross-correlation model (deep-CCM) is constructed to not only model the extracted feature hierarchies of CNN independently, but also relate the statistical dependencies of paired features from different layers. Then, a Bayesian error weighting scheme for a classifier combination is adopted to explore the contributions from individual Deep-CCM classifiers to enhance the accuracy of shot classification in each image frame. Third, we feed the frame-wise classification results to a linear-chain conditional random field module to refine the shot predictions by taking into account the global and temporal regularities. We provide extensive experimental results on a data set of live concert videos to demonstrate the advantage of the proposed CC-Net over existing popular fusion approaches for shot classification. [ABSTRACT FROM AUTHOR]

Interaction Style (IS) refers to patterns of interaction containing highly contextual and innate information. Awareness of our IS can help us discover interpersonal conflicts and guide us how to interact with others. Recently, automatic IS recognition is becoming increasingly important in the design of a dialogue system for harmonious interaction. With the goal to select appropriate responses, four IS types proposed by Berens are selected as the basis for our study. In this study, multiple views (multi-views) of the utterances during interaction, including emotions and dialogue topics, are recognized first. Inspired by the emotion profile theory, the IS profiles are then extracted using the multi-view features to better characterize the IS of the interactional utterances. Similar to the multilayer architectures in deep neural networks, a multi-layer multi-view IS profile representation method, structured layer by layer through embedding the multi-views, is proposed to better interpret intermediate representations in the feature space of the interactional utterances based on a probabilistic fusion model. The IS is finally recognized by using the Support Vector Machine (SVM) based on the obtained IS profiles. Experimental results demonstrate that the proposed method achieved an encouraging IS recognition accuracy and outperformed the previous method. [ABSTRACT FROM PUBLISHER]

This paper aimed to explore the mechanical properties of a cement-based material with carbon nanotube (CNT) under drying and freeze-thaw environments. Mercury Intrusion Porosimetry and Scanning Electron Microscopy were used to analyze the pore structure and microstructure of CNT/cement composite, respectively. The experimental results showed that multi-walled CNT (MWCNT) could improve to different degrees the mechanical properties (compressive and flexural strengths) and physical performances (shrinkage and water loss) of cement-based materials under drying and freeze-thaw conditions. This paper also demonstrated that MWCNT could interconnect hydration products to enhance the performance of anti-microcracks for cement-based materials, as well as the density of materials due to CNT's filling action. [ABSTRACT FROM AUTHOR]

Speaking effect is a crucial issue that may dramatically degrade performance in emotion recognition from facial expressions. To manage this problem, an eigenface conversion-based approach is proposed to remove speaking effect on facial expressions for improving accuracy of emotion recognition. In the proposed approach, a context-dependent linear conversion function modeled by a statistical Gaussian Mixture Model (GMM) is constructed with parallel data from speaking and non-speaking facial expressions with emotions. To model the speaking effect in more detail, the conversion functions are categorized using a decision tree considering the visual temporal context of the Articulatory Attribute (AA) classes of the corresponding input speech segments. For verification of the identified quadrant of emotional expression on the Arousal-Valence (A-V) emotion plane, which is commonly used to dimensionally define the emotion classes, from the reconstructed facial feature points, an expression template is constructed to represent the feature points of the non-speaking facial expressions for each quadrant. With the verified quadrant, a regression scheme is further employed to estimate the A-V values of the facial expression as a precise point in the A-V emotion plane. Experimental results show that the proposed method outperforms current approaches and demonstrates that removing the speaking effect on facial expression is useful for improving the performance of emotion recognition. [ABSTRACT FROM PUBLISHER]

A complete emotional expression typically contains a complex temporal course in face-to-face natural conversation. To address this problem, a bimodal hidden Markov model (HMM)-based emotion recognition scheme, constructed in terms of sub-emotional states, which are defined to represent temporal phases of onset, apex, and offset, is adopted to model the temporal course of an emotional expression for audio and visual signal streams. A two-level hierarchical alignment mechanism is proposed to align the relationship within and between the temporal phases in the audio and visual HMM sequences at the model and state levels in a proposed semi-coupled hidden Markov model (SC-HMM). Furthermore, by integrating a sub-emotion language model, which considers the temporal transition between sub-emotional states, the proposed two-level hierarchical alignment-based SC-HMM (2H-SC-HMM) can provide a constraint on allowable temporal structures to determine an optimal emotional state. Experimental results show that the proposed approach can yield satisfactory results in both the posed MHMC and the naturalistic SEMAINE databases, and shows that modeling the complex temporal structure is useful to improve the emotion recognition performance, especially for the naturalistic database (i.e., natural conversation). The experimental results also confirm that the proposed 2H-SC-HMM can achieve an acceptable performance for the systems with sparse training data or noisy conditions. [ABSTRACT FROM PUBLISHER]

This paper presents an approach to the automatic recognition of human emotions from audio-visual bimodal signals using an error weighted semi-coupled hidden Markov model (EWSC-HMM). The proposed approach combines an SC-HMM with a state-based bimodal alignment strategy and a Bayesian classifier weighting scheme to obtain the optimal emotion recognition result based on audio-visual bimodal fusion. The state-based bimodal alignment strategy in SC-HMM is proposed to align the temporal relation between audio and visual streams. The Bayesian classifier weighting scheme is then adopted to explore the contributions of the SC-HMM-based classifiers for different audio-visual feature pairs in order to obtain the emotion recognition output. For performance evaluation, two databases are considered: the MHMC posed database and the SEMAINE naturalistic database. Experimental results show that the proposed approach not only outperforms other fusion-based bimodal emotion recognition methods for posed expressions but also provides satisfactory results for naturalistic expressions. [ABSTRACT FROM PUBLISHER]