Search

Micro-expression is a spontaneous expression that occurs when a person tries to mask his or her inner emotion, and can neither be forged nor suppressed. It is a kind of short-duration, low-intensity, and usually local-motion facial expression. However, owing to these characteristics of micro-expression, it is difficult to obtain micro-expression data, which is the bottleneck of applying deep learning methods to micro-expression recognition. In addition, micro-expression is still a type of expression, and it can also be encoded by the facial action coding system. Therefore, there is a certain correlation between action unit recognition and micro-expression recognition. Addressing those, we propose a novel knowledge transfer technique distills and transfers knowledge from action unit for micro-expression recognition, where knowledge from a pre-trained deep teacher neural network is distilled and transferred to a shallow student neural network. Specifically, a teacher-student correlative framework is designed with a novel objective function. And features extracted from the teacher network is used as prior knowledge to guide the student part to efficiently learning from the target micro-expression dataset. Experiments are conducted on four available published micro-expression datasets (SMIC2, CASME, CASME II and SAMM). The experimental results show that our model outperforms the state-of-the-art systems

This paper introduces a pose error compensation method performed in joint space to improve both position and orientation accuracy of industrial robots. Joint space division is proposed to achieve dimension reduction to reduce the workload of pose error sampling. The six-dimensional joint space is divided into two three-dimensional subspaces by the wrist center. Each subspace is discretized by a sequence of grid elements of which grid vertices are considered as sample points. Based on point set registration, two spatial databases of pose errors are built in the two subspaces respectively using a laser tracker. For a given robot posture, the pose errors generated in the two subspaces can be separately estimated using spatial interpolation based on inverse distance weighting algorithm. Formed by the estimated values, two transformation error matrices are introduced to compensate for the pose errors of the two subspaces. A Comau NJ370–3.0 manipulator was employed as an experimental platform to evaluate the effectiveness of the proposed method. Experimental results demonstrated the maximum position error was reduced by 96.06% to below 0.334 mm, and the maximum orientation errors have decreased to below 0.027° after compensation was performed.

This paper introduces the usage of semi-supervised learning to obtain competitive detection accuracy of measuring drilled holes on composite parts with very limited noisy training data. An improved texture segmentation algorithm based on local binary patterns algorithm is proposed, named local exponential patterns. The algorithm divides the image texture into nine levels, of which the highest level of texture is selected for contour extraction. An ellipse fitting method is used to fit the target contours and vote for the candidate ellipses. The regions inside the candidate ellipses are taken as the semi-supervised semantic label for images. A new loss named round loss is proposed, and a superior circle segmentation model was trained by learning from incompletely annotated data. To verify the effectiveness of the method, experiments were conducted with the drilled holes on the composite parts. The results show that the proposed semi-supervised deep learning approach is exceedingly suitable for circle detection of holes with different texture information commonly found in robotic drilling. Massive data labeling can be completely avoided with proposed method. The measurement accuracy can reach 0.03 mm, which can meet the visual measurement requirements of the circular holes on composite parts in the robotic drilling system.

Automatic emotion recognition is a challenging task that can make great impact on improving natural human-computer interactions. In this paper, we present our automatic prediction of dimensional emotional state for Cross-cultural Emotion Sub-Challenge (AVEC 2018), which uses multi-features and fusion across visual, audio and text modalities. Single-feature predictions are modeled at first with support vector regression (SVR). The multimodal fusion of these modalities is then performed with a multiple linear regression model. Besides the baseline features, we extract one-gram and two-gram features from text, and some types of convolutional neural networks (CNNs) feature from video. Our multimodal fusion reached CCC=0.599 on the development set for arousal, 0.617 for valence and 0.289 for likability.

Micro-expression is a special facial movement, which can be used as an important basis for judging people's subjective emotions. Constrained by the physiology, micro-expression can be described temporally by four phases: neutral, onset, apex, and offset. And previous studies confirmed that using the crucial temporal sequences is better than using the whole video for micro-expression recognition. Therefore, micro-expression movement spotting is considered beneficial for micro-expression recognition. While it is a challenging task due to the short duration, low intensity and usually local motion characteristics of micro-expression. Inspired by the mechanism of the ventral and dorsal visual pathways in the cerebral visual cortex, we propose an end2end two-stream attention-aware network for micro-expression movement spotting in this paper. We construct a spatial-temporal cascaded network for each stream which combines convolutional neural network and attention-aware bilateral long short-term memory recurrent neural network. And we apply the attention mechanism to two-stream feature fusion. Experiments are conducted on three available published micro-expression datasets (SMIC2, CASME, and CASME II). The experimental results show that the proposed framework outperforms state-of-the-art methods for the task of micro-expression movement spotting.

Integrated Energy System (IES) based on Combined Cooling, Heating and Power (CCHP) technology has been proved as an efficient approach, which can resolve energy and environment issues and meanwhile bring economic benefits. For areas with fragile grid infrastructure and abundant oil, gas and renewable resources, various types of energy sources can complement each other and be cooperated as a unity, by operating the energy networks across regions as a single IES through the implementation of multi-region Gas and Power Complementary (GPC) system. Conventionally, the typical optimized scheduling of GPC system does not consider the complementarity among areas with diverse device and load characteristics. Since changes and developments have occurred in power-to-gas conversion technologies and intra-regional CCHP system, traditional models need to be updated and improved along with the application of tiered gas price. Based on the existing coupled power-gas flow scheduling model, this paper proposed a multi-region optimized scheduling model where the CCHP systems operated inside a region and power-gas complementarity could be achieved among multiple regions. Mixed Integer Linear Programming (MILP) was used to obtain the optimal solution of this scheduling model. The compressor linearization model was designed in order to describe transmission loss of the gas network and perfect the power-gas flow model as well. Results of cases proved that considerable environmental benefits could be achieved when Combined Heating and Power (CHP) equipment was introduced in the multi-region GPC system. Finally, combined with the tiered gas tariff, the total operation cost of regions could be reduced and the day-ahead economic optimized scheduling could be improved. It also showed that the operating parameters in each region expressed different sensitivity to gas price due to the flow constraints. Moreover, compared with battery energy storage systems, power-to-gas conversion technology would be more prospective because it could significantly boost the system operating efficiency.

Affect presentation is periodic and multi-modal, such as through facial movements, body gestures, and so on. Studies have shown that temporal selection and multi-modal combinations may benefit affect recognition. In this article, we therefore propose a spatio-temporal fusion model that extracts spatio-temporal hierarchical features based on select expressive components. In addition, a multi-modal hierarchical fusion strategy is presented. Our model learns the spatio-temporal hierarchical features from videos by a proposed deep network, which combines a convolutional neural networks (CNN), bilateral long short-term memory recurrent neural networks (BLSTM-RNN) with principal component analysis (PCA). Our approach handles each video as a “video sentence.” It first obtains a skeleton with the temporal selection process and then segments key words with a certain sliding window. Finally, it obtains the features with a deep network comprised of a video-skeleton and video-words. Our model combines the feature level and decision level fusion for fusing the multi-modal information. Experimental results showed that our model improved the multi-modal affect recognition accuracy rate from 95.13% in existing literature to 99.57% on a face and body (FABO) database, our results have been increased by 4.44%, and it obtained a macro average accuracy (MAA) up to 99.71%.

This paper presents our work in the FG 2017 Facial Expression Recognition and Analysis challenge (FERA 2017) and we participate in the AU occurrence sub-challenge. Our work of AU occurrence recognition is based on deep learning, and we design convolution neural network (CNN) models for two types of work: facial view recognition and AU occurrence recognition. For facial view recognition, our model could achieve 97.7% accuracy on validation dataset about 9 facial views. For AU occurrence recognition, we use both visual features and temporal information of dataset. We use CNN models to get deep visual feature and then use BLSTM-RNN to learn the high-level feature in the time domain. When training models, we divide dataset into 9 parts based on 9 facial views, and each model is trained in a specific view. When recognizing AUs, we recognize facial view first and then choose the corresponding model for AU occurrence recognition. Finally, our method shows good performance, the F1 score of test data is 0.507 and the accuracy is 0.735.

This paper presents our work in the Emotion Sub-Challenge of the 6th Audio/Visual Emotion Challenge and Workshop (AVEC 2016), whose goal is to explore utilizing audio, visual and physiological signals to continuously predict the value of the emotion dimensions (arousal and valence). As visual features are very important in emotion recognition, we try a variety of handcrafted and deep visual features. For each video clip, besides the baseline features, we extract multi-scale Dense SIFT features (MSDF), and some types of Convolutional neural networks (CNNs) features to recognize the expression phases of the current frame. We train linear Support Vector Regression (SVR) for every kind of features on the RECOLA dataset. Multimodal fusion of these modalities is then performed with a multiple linear regression model. The final Concordance Correlation Coefficient (CCC) we gained on the development set are 0.824 for arousal, and 0.718 for valence; and on the test set are 0.683 for arousal and 0.642 for valence.

Face detection is important for face localization in face or facial expression recognition, etc. The basic idea is to determine whether there is a face in an image or not, and also its location, size. It can be seen as a binary classification problem, which can be well solved by support vector machine (SVM). Though SVM has strong model generalization ability, it has some limitations, which will be deeply analyzed in the paper. To access them, we study the principle and characteristics of the Multiple Kernel Learning (MKL) and propose a MKL-based face detection algorithm. In the paper, we describe the proposed algorithm in the interdisciplinary research perspective of machine learning and image processing. After analyzing the limitation of describing a face with a single feature, we apply several ones. To fuse them well, we try different kernel functions on different feature. By MKL method, the weight of each single function is determined. Thus, we obtain the face detection model, which is the kernel of the proposed method. Experiments on the public data set and real life face images are performed. We compare the performance of the proposed algorithm with the single kernel-single feature based algorithm and multiple kernels-single feature based algorithm. The effectiveness of the proposed algorithm is illustrated. Keywords: face detection, feature fusion, SVM, MKL

Human emotions are known to always have four phases in the temporal domain: neutral, onset, apex, and offset. This has been demonstrated to be of great benefit for emotion recognition. Therefore, temporal segmentation has attracted considerable research interest. Although state-of-the-art techniques use recurrent neural networks to highly increase the performance, they ignore the relevance of each frame (time step) of a video, and they do not consider the changing contribution of different features when fusing them. We propose a framework called dual-level attention-aware bidirectional grated recurrent unit, which integrates ideas from attention models to discover the most important frames and features for improving temporal segmentation. Specifically, it applies attention mechanisms at two levels: frame and feature. A significant advantage is that the two-level attention weights provide a meaningful value to depict the importance of each frame and feature. The experiments demonstrated that the proposed framework outperforms state-of-the-art methods.

Constrained by the physiology, the temporal factors associated with human behavior, irrespective of facial movement or body gesture, are described by four phases: neutral, onset, apex, and offset. Although they may benefit related recognition tasks, it is not easy to accurately detect such temporal segments. An automatic temporal segment detection framework using bilateral long short-term memory recurrent neural networks (BLSTM-RNN) to learn high-level temporal–spatial features, which synthesizes the local and global temporal–spatial information more efficiently, is presented. The framework is evaluated in detail over the face and body database (FABO). The comparison shows that the proposed framework outperforms state-of-the-art methods for solving the problem of temporal segment detection.