Search

Emotion recognition plays an important role in several applications, such as human computer interaction and understanding affective state of users in certain tasks, e.g., within a learning process, monitoring of elderly, interactive entertainment etc. It may be based upon several modalities, e.g., by analyzing facial expressions and/or speech, using electroencephalograms, electrocardiograms etc. In certain applications the only available modality is the user's (speaker's) voice. In this paper we aim to analyze speakers' emotions based solely on paralinguistic information, i.e., not depending on the linguistic aspect of speech. We compare two machine learning approaches, namely a Convolutional Neural Network and a Support Vector Machine. The former is trained using raw speech information, while the latter is trained on a set of extracted low-level features. Aiming to provide a multilingual approach, training and testing datasets contain speech from different languages.

Emotion recognition from speech may play a crucial role in many applications related to human–computer interaction or understanding the affective state of users in certain tasks, where other modalities such as video or physiological parameters are unavailable. In general, a human’s emotions may be recognized using several modalities such as analyzing facial expressions, speech, physiological parameters (e.g., electroencephalograms, electrocardiograms) etc. However, measuring of these modalities may be difficult, obtrusive or require expensive hardware. In that context, speech may be the best alternative modality in many practical applications. In this work we present an approach that uses a Convolutional Neural Network (CNN) functioning as a visual feature extractor and trained using raw speech information. In contrast to traditional machine learning approaches, CNNs are responsible for identifying the important features of the input thus, making the need of hand-crafted feature engineering optional in many tasks. In this paper no extra features are required other than the spectrogram representations and hand-crafted features were only extracted for validation purposes of our method. Moreover, it does not require any linguistic model and is not specific to any particular language. We compare the proposed approach using cross-language datasets and demonstrate that it is able to provide superior results vs. traditional ones that use hand-crafted features.

Emotion recognition plays an important role in several applications, such as human computer interaction and understanding affective state of users in certain tasks, e.g., within a learning process, monitoring of elderly, interactive entertainment etc. It may be based upon several modalities, e.g., by analyzing facial expressions and/or speech, using electroencephalograms, electrocardiograms etc. In certain applications the only available modality is the user’s (speaker’s) voice. In this paper we aim to analyze speakers’ emotions based solely on paralinguistic information, i.e., not depending on the linguistic aspect of speech. We compare two machine learning approaches, namely a Convolutional Neural Network and a Support Vector Machine. The former is trained using raw speech information, while the latter is trained on a set of extracted low-level features. Aiming to provide a multilingual approach, training and testing datasets contain speech from different languages.

This paper presents an Internet-of-Things approach to fusing audio sensors towards the detection of audio events, in a meeting room scenario. The different types of audio sensors (microphones) and respective individual audio analysis modules are incorporated within the context of an IoT framework that follows a message-oriented architecture. Each individual audio analysis module is composed by a feature extraction stage and a Support-Vector-Machine (SVM) classifier. A fusion module is also adopted to combine the individual sensor-level decisions, in order to extract the final classification decision. A detailed experimental evaluation on a publicly available real-world dataset proves a rather significant performance boosting in terms of overall classification accuracy. In addition, the proposed architecture enables an easy-to-use training procedure that can easily handle any number of audio sensors and respective classifiers without any prior knowledge of the room's geometry or any other constraints regarding the topological condition of the sensors.

In this paper we present SYNAISTHISI, i.e., a cloud-based platform, that provides the necessary infrastructure in order to interconnect heterogeneous devices and services over heterogeneous networks. SYNAISTHISI facilitates the orchestration of a collective functionality allowing several services to be managed through agents that dynamically allocate the available resources.. In a smart room use-case, multiple sensing and actuation units have been developed and deployed in a lecture room. Maintaining comfort levels for room occupants is achieved through automatic decision making that exploits information from a complex event recognition engine. Our goal is to improve the overall working environment, while minimizing energy losses.

Changing clothes is a basic activity of daily living (ADL) which may be used as a measurement of the functional status of e.g. an elderly person, or a person with certain disabilities. In this paper we propose a methodology for the detection of when a human has changed clothes. Our non-contact unobtrusive monitoring system is built upon the Microsoft Kinect depth camera. It uses the OpenNI SDK to detect a human skeleton and extract the upper and lower clothes’ visual features. Color, texture and edge descriptors are then extracted and fused. We evaluate our system on a publicly available set of real recordings for several users and under various illumination conditions. Our results show that our system is able to successfully detect when a user changes clothes, thus to assess the quality of the corresponding ADL.

The purpose of the research described in this paper is to examine the existence of correlation between low level audio, visual and textual features and movie content similarity. In order to focus on a well defined and controlled case, we have built a small dataset of movie scenes from three sequel movies. In addition, manual annotations have led to a ground-truth similarity matrix between the adopted scenes. Then, three similarity matrices (one for each medium) have been computed based on Gaussian Mixture Models (audio and visual) and Latent Semantic Indexing (text). We have evaluated the automatically extracted similarities along with two simple fusion approaches and results indicate that the low-level features can lead to an accurate representation of the movie content. In addition, the fusion approach seems to outperform the individual modalities, which is a strong indication that individual modules lead to diverse similarities (in terms of content). Finally, we have evaluated the extracted similarities for different groups of human annotators, based on what a human interprets as similar and the results show that different groups of people correlate better with different modalities. This last result is very important and can be either used in (a) a personalized content-based retrieval and recommender system and (b) in a local weighted fusion approach, in future research.