Search

Sparse Distributed Memory (SDM) is a mathematical associative human long-term memory model that is psychologically and neuroscientifically plausible. It is content addressable and can potentially store unlimited high dimensional information although it only accepts binary vectors. Although an integer SDM has been proposed to store non-binary vectors, the size of the model has been increased significantly. In this paper, we propose a biologically plausible encoding scheme to enable SDM to accept integer vectors, requiring much less space than the existing integer SDM model. A public handwritten digits dataset with integer values has been used for the evaluation of our proposed model and satisfied performance has been achieved in both data retrieval and pattern recognition.

Holoscopic 3D imaging system, also known as integral imaging, is an innovative 3D imaging principle that overcomes key limitation of traditional 2D imaging issues such as depth, scalability and multi-perspective with its simplistic form of 3D data acquisition and visualisation which provides robust and spatial information of the real world 3D scene. In this paper, an innovative 3D map generation technique is proposed which produces accurate 3D map of a scene from a single holoscopic 3D image based on angular information preserved in its elemental images. 3D depth map is generated from the elemental images based on the semi-global block-based matching algorithm. A weighted least squares filter is utilised to minimise the noise in the resulting disparity image. The evaluation result outperformed state of the art 3D depth generation techniques and the proposed technique enlarges the industrial application of 3D imaging applications such as AR/VR, inspection, robotics, security and entertainment.

Since signal-to-noise ratio and axial resolution are effective decision parameters in ultrasound imaging and ultrasonic testing system, we combined nonlinear frequency modulation and Barker code sequence to improve these parameters, thereby improving the effect and performance in the ultrasound imaging and ultrasonic testing system. The theoretical study on the signal generation and decoding has been presented and the cyst phantom simulation has been carried out. The theoretical analysis shows that the new code method can improve 8.46 dB in Contrast ratio compared to simple pulse signal. The simulation produced the similar results.

Traditional fuzzy clustering algorithms suffer from two problems in image segmentations. One is that these algorithms are sensitive to outliers due to the non-sparsity of fuzzy memberships. The other is that these algorithms often cause image over-segmentation due to the loss of image local spatial information. To address these issues, we propose a robust self-sparse fuzzy clustering algorithm (RSSFCA) for image segmentation. The proposed RSSFCA makes two contributions. The first concerns a regularization under Gaussian metric that is integrated into the objective function of fuzzy clustering algorithms to obtain fuzzy membership with sparsity, which reduces a proportion of noisy features and improves clustering results. The second concerns a connected-component filtering based on area density balance strategy (CCF-ADB) that is proposed to address the problem of image over-segmentation. Compared to the integration of local spatial information into the objective functions, the presented CCF-ADB is simpler and faster for the removal of small areas. Experimental results show that the proposed RSSFCA addresses two problems in current fuzzy clustering algorithms, i.e., the outlier sensitivity and the over-segmentation, and it provides better image segmentation results than state-of-the-art algorithms. 10.13039/501100001809-National Natural Science Foundation of China (Grant Number: 61871259, 61811530325 (International Exchange Cooperation (IEC)\NSFC\170396, Royal Society, U.K.), 61871260, 61672333 and 61861024)

The V-Net based 3D fully convolutional neural networks have been widely used in liver volumetric data segmentation. However, due to the large number of parameters of these networks, 3D FCNs suffer from high computational cost and GPU memory usage. To address these issues, we design a lightweight V-Net (LV-Net) for liver segmentation in this paper. The proposed network makes two contributions. The first is that we design an inverted residual bottleneck block (IRB block) and a 3D average pooling block and apply them to the proposed LV-Net. Compared with vanilla convolution, depth-wise convolution and point-wise convolution employed by the IRB block can not only reduce the number of parameters significantly, but also extract features sufficiently well by decoupling cross-channel corrections and spatial correlations. The second is that the LV-Net employs 3D deep supervision to improve the final loss function in training phase, which makes the proposed LV-Net acquire a more powerful discrimination capability between liver areas and non-liver areas. The proposed LV-Net is evaluated on public LiTS dataset, and experiments demonstrate that the proposed LV-Net is superior to popular 2D and 3D networks in terms of segmentation performance, parameter quantity and computational cost.

© 2020 by the authors. Wireless Capsule Endoscopy is a state-of-the-art technology for medical diagnoses of gastrointestinal diseases. The amount of data produced by an endoscopic capsule camera is huge. These vast amounts of data are not practical to be saved internally due to power consumption and the available size. So, this data must be transmitted wirelessly outside the human body for further processing. The data should be compressed and transmitted efficiently in the domain of power consumption. In this paper, a new approach in the design and implementation of a low complexity, multiplier-less compression algorithm is proposed. Statistical analysis of capsule endoscopy images improved the performance of traditional lossless techniques, like Huffman coding and DPCM coding. Furthermore the Huffman implementation based on simple logic gates and without the use of memory tables increases more the speed and reduce the power consumption of the proposed system. Further analysis and comparison with existing state-of-the-art methods proved that the proposed method has better performance.

Face recognition has made significant progress in recent years. It has been successfully applied to many applications from security entrance systems for staff identification to visual surveillance. However, the performance of current systems are highly affected by factors such as subject’s head orientation, facial expressions, and face wear-on. In order to address this challenge, researchers usually resort to more training data or more complex recognition models. We believe the secret lies in using a more rich data source. Therefore, in this paper, we use a holoscopic 3D (H3D) imaging system. To show the effectiveness of the sensor, we use it with a simple back propagation neural network (BPNN) classifier. We illustrate the accuracy of the system through an experiment and show that it has high accuracy and is robust against challenges aforementioned.

With the rapid development of deep learning (DL) in recent years, automatic modulation recognition (AMR) with DL has achieved high accuracy. However, aiming to obtain higher classification accuracy, DL requires numerous training samples. In order to solve this problem, it is a challenge to study how to efficiently use DL for AMR in the case of few samples. In this letter, inspired by the capsule network (CapsNet), we propose a new network structure named AMR-CapsNet to achieve higher classification accuracy of modulation signals with fewer samples, and further analyze the adaptability of DL models in the case of few samples. The simulation results demonstrate that when 3% of the dataset is used to train and the signal-to-noise ratio (SNR) is greater than 2 dB, the overall classification accuracy of the AMR-CapsNet is greater than 80%. Compared with convolutional neural network (CNN), the classification accuracy is improved by 20%.

The EmoPain 2020 Challenge is the first international competition aimed at creating a uniform platform for the comparison of machine learning and multimedia processing methods of automatic chronic pain assessment from human expressive behaviour, and also the identification of pain-related behaviours. The objective of the challenge is to promote research in the development of assistive technologies that help improve the quality of life for people with chronic pain via real-time monitoring and feedback to help manage their condition and remain physically active. The challenge also aims to encourage the use of the relatively underutilised, albeit vital bodily expression signals for automatic pain and pain-related emotion recognition. This paper presents a description of the challenge, competition guidelines, bench-marking dataset, and the baseline systems' architecture and performance on the three sub-tasks: pain estimation from facial expressions, pain recognition from multimodal movement, and protective movement behaviour detection., Comment: 8 pages

Recently, video-based micro-gesture recognition with the data captured by holoscopic 3D (H3D) sensors is getting more and more attention, mainly because of their particular advantages to use a single aperture camera to embed the 3D information in 2D images. However, it is not easy to use the embedded 3D information in an efficient manner due to the special imaging principles of H3D sensors. In this paper, an efficient Pseudo View Points (PVP) based method is proposed to introduce the embedded 3D information in H3D images into a new micro-gesture recognition framework. Specifically, we obtain several pseudo view points based frames by composing all the pixels at the same position in each elemental image(EI) in the original H3D frames. This is a very efficient and robust step, and could mimic the real view points so as to represent the 3D information in the frames. Then, a new recognition framework based on 3D DenseNet and Bi-GRU networks is proposed to learn the dynamic patterns of different micro-gestures based on the representation of the pseudo view points. Finally, we perform a thorough comparison of the related benchmark, which demonstrates the effectiveness of our method and also reports a new state of the art performance.

Finger microgestures have been widely used in human computer interaction (HCI), particularly for interactive applications, such as virtual reality (VR) and augmented reality (AR) technologies, to provide immersive experience. However, traditional 2D image-based microgesture recognition suffers from low accuracy due to the limitations of 2D imaging sensors, which have no depth information. In this article, we proposed an innovative 3D microgesture recognition system based on a holoscopic 3D imaging sensor. Due to the lack of holoscopic 3D datasets, a comprehensive holoscopic 3D microgesture (HoMG) database is created and used to develop a robust 3D microgesture recognition method. Then, a fast algorithm is proposed to extract multiviewpoint images from one holoscopic image. Furthermore, we applied a CNN model with an attention-based residual block to each viewpoint image to improve the algorithm performance. Finally, bagging classification tree decision-level fusion is applied to combine the predictions. The experimental results demonstrate that the proposed method outperforms state-of-the-art methods and delivers a better accuracy than existing methods.

Immersive technology attempts to emulate a physical world through the means of a digital or simulated world. Micro-gestures are small variation actions on human hands defined by user that is one of the most convenient human action in immersive technology. Holoscopic 3D imaging uses bionics technology to capture spatial image in the pattern of “fly’s eye” and it has fruitful 3D cubic information compared to 2D images that can be used for high accurate micro-gesture controller systems. In this paper, a new micro-gesture recognition system based on holoscopic 3D imaging system is proposed for immersive applications. It is built on fast pre-processing, dynamic image feature extraction and a non-linear Support Vector Machine classifier. It is evaluated on the public Holoscopic Micro 3D Gesture (HoMG) dataset outperforming all the existing state-of-the-art methods on the same dataset.

Precisely describing the action inside of a video is a challenging task because the content of the video includes various objects, with different local motion information at different speed in the video frames. In this paper, a new video feature is proposed based on the spatial information of the objects in a frame, along with the motion information between one against multiple consecutive frames. Motion information between pixels at the same position in the whole video are all combined for a new Spatial Multi-Scale Motion History Histogram (SMMHH) dynamic descriptor. The detailed algorithm of the SMMHH was given and it is tested in both human action recognition and touch gesture recognition applications based on the public video datasets. Experimental results demonstrate its excellent performance compared to other traditional methods.

Morphological reconstruction (MR) is often employed by seeded image segmentation algorithms such as watershed transform and power watershed as it is able to filter seeds (regional minima) to reduce over-segmentation. However, MR might mistakenly filter meaningful seeds that are required for generating accurate segmentation and it is also sensitive to the scale because a single-scale structuring element is employed. In this paper, a novel adaptive morphological reconstruction (AMR) operation is proposed that has three advantages. Firstly, AMR can adaptively filter useless seeds while preserving meaningful ones. Secondly, AMR is insensitive to the scale of structuring elements because multiscale structuring elements are employed. Finally, AMR has two attractive properties: monotonic increasingness and convergence that help seeded segmentation algorithms to achieve a hierarchical segmentation. Experiments clearly demonstrate that AMR is useful for improving algorithms of seeded image segmentation and seed-based spectral segmentation. Compared to several state-of-the-art algorithms, the proposed algorithms provide better segmentation results requiring less computing time. Source code is available at https://github.com/SUST-reynole/AMR.

Copyright © 2019 The authors. Ultrasonic Testing (UT) is one of the most important technologies in Non-Detective Testing (NDT) methods. Recently, Barker code and Golay code pairs as coded excitation signals have been applied in ultrasound imaging system with improved quality. However, the signal-to-noise ratio (SNR) of existing UT system based on Barker code or Golay code can be influenced under high high attenuation materials or noisy conditions. In this paper, we apply the convolution of Barker and Golay codes as coded excitation signals for low voltage UT devices that combines the advantages of Barker code and Golay code together. There is no need to change the hardware of UT system in this method. The proposed method has been analyzed theoretically and then in extensive simulations. The experimental results demonstrated that the main lobe level of the code produced by convolution of Barker code and Golay code pairs is much higher than the simple pulse and the main lobe of the combined code is higher than the traditional Barker code, sidelobe is the same as the baker code that constitutes this combined code. So the peak sidelobe level (PSL) of the combined code is lower than the traditional Barker code. Equipped with this, UT devices can be applied in low voltage situations.

Active thermography provides infrared images that contain sub-surface defect information, while visible images only reveal surface information. Mapping in- frared information to visible images o ers more comprehensive visualization for decision-making in rail inspection. However, the common information for regis- tration is limited due to di erent modalities in both local and global level. For example, rail track which has low temperature contrast reveals rich details in visible images, but turns blurry in the infrared counterparts. This paper pro- poses a registration algorithm called Edge-Guided Speeded-Up-Robust-Features (EG-SURF) to address this issue. Rather than sequentially integrating local and global information in matching stage which su ered from buckets e ect, this al- gorithm adaptively integrates local and global information into a descriptor to gather more common information before matching. This adaptability consists of two facets, an adaptable weighting factor between local and global information, and an adaptable main direction accuracy. The local information is extracted using SURF while the global information is represented by shape context from edges. Meanwhile, in shape context generation process, edges are weighted ac- cording to local scale and decomposed into bins using a vector decomposition manner to provide more accurate descriptor. The proposed algorithm is qual- itatively and quantitatively validated using eddy current pulsed thermography scene in the experiments. In comparison with other algorithms, better perfor- mance has been achieved. This work is supported by Engineering and Physical Sciences Research Coun- cil (EPSRC) [grant numbers EP/K503885/1]. Chaoqing Tang would also like to thank China Scholarship Council for funding his Ph.D study.

Pain -related emotions are a major barrier to effective self rehabilitation in chronic pain. Automated coaching systems capable of detecting these emotions are a potential solution. This paper lays the foundation for the development of such systems by making three contributions. First, through literature reviews, an overview of how chronic pain is expressed and the motivation for detecting it in physical rehabilitation is provided. Second, a fully labelled multimodal dataset containing high resolution multiple-view face videos, head mounted and room audio signals, full body 3-D motion capture and electromyographic signals from back muscles is supplied. Natural unconstrained pain related facial expressions and body movement behaviours were elicited from people with chronic pain carrying out physical exercises. Both instructed and non-instructed exercises where considered to reflect different rehabilitation scenarios. Two sets of labels were assigned: level of pain from facial expressions annotated by eight raters and the occurrence of six pain-related body behaviours segmented by four experts. Third, through exploratory experiments grounded in the data, the factors and challenges in the automated recognition of such expressions and behaviour are described, the paper concludes by discussing potential avenues in the context of these findings also highlighting differences for the two exercise scenarios addressed. This work was funded by the EPSRC Emotion & Pain Project (EP/H017178/1, EP/H017194/1, EP/H016988/1).

In this paper, we propose a novel approach based on a symmetric fully convolutional network within pyramid pooling (FCN-PP) for landslide mapping (LM). The proposed approach has three advantages. Firstly, this approach is automatic and insensitive to noise because multivariate morphological reconstruction (MMR) is used for image preprocessing. Secondly, it is able to take into account features from multiple convolutional layers and explore efficiently the context of images, which leads to a good tradeoff between wider receptive field and the use of context. Finally, the selected pyramid pooling module addresses the drawback of single-scale pooling employed by convolutional neural network (CNN), fully convolutional network (FCN), U-Net, etc. Experimental results show that the proposed FCN-PP is effective for LM, and it outperforms state-of-the-art approaches in terms of four metrics, Precision, Recall, F -score, and Accuracy.

Three-dimensional (3D) convolutional networks have been proven to be able to explore spatial context and spectral information simultaneously for super-resolution (SR). However, such kind of network can’t be practically designed very ‘deep’ due to the long training time and GPU memory limitations involved in 3D convolution. Instead, in this paper, spatial context and spectral information in hyperspectral images (HSIs) are explored using Two-dimensional (2D) and One-dimenional (1D) convolution, separately. Therefore, a novel 2D-1D generative adversarial network architecture (2D-1D-HSRGAN) is proposed for SR of HSIs. Specifically, the generator network consists of a spatial network and a spectral network, in which spatial network is trained with the least absolute deviations loss function to explore spatial context by 2D convolution and spectral network is trained with the spectral angle mapper (SAM) loss function to extract spectral information by 1D convolution. Experimental results over two real HSIs demonstrate that the proposed 2D-1D-HSRGAN clearly outperforms several state-of-the-art algorithms.

Active Shape Model (ASM) has been successfully applied in the segmentation of Diffusion Tensor Magnetic Resonance Image (DT-MRI, referred to as DTI) of brain. However, due to multiple anatomical structure types, irregular shapes, small gray-scale and large amount of these images, perfect segmentation performance could not be achieved. Especially, it is sensitive to initial values with high computational complexity. In this paper, we introduce the gray information of multiple atlases and the prior information of target shapes into the ASM and propose the Multi-Atlas Active Shape Model (referred to as MA-ASM) approach for DTI segmentation. It was evaluated in a manually labeled database with 7 Region of Interest (ROI)s for each of 20 subjects. In comparison with the state of art method of STAPLE (Simultaneous Truth Performance Level Estimation), the proposed algorithm was closer to the manual segmentation shape by subjective visual effects, and had higher overlap rates and lower error detection rates on quantitative analysis than STAPLE.

Because emotions are such a significant and integral part of being human, understanding them and knowing how to react to the emotions of others is a fundamental requirement for successful social interaction. We recognize emotions primarily through speech and facial expression. The topic is gaining importance in academic research triggered by research in new techniques such as identifying emotions based on speech context. This investigates the relationship between emotions and the content of our speech. This paper proposes how emotion in speech and facial expression can be recognized in real time using a framework consisting of mobile phone technology backed by cloud computing. This functionality was developed and built into a mobile phone application. Currently, the application works on any Android smartphone to detect and identify emotions in real time. The results are expressed as a percentage of all the possible emotions, such as sad, happy, fear, surprise, anger and so on. The results of the experiment confirm that face and speech emotion recognition was conducted successfully using a smartphone. It was correct in 97.26% of instances when used with standard corpora: a.Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS) b.the Surrey Audio-Visual Expressed Emotion (SAVEE)

The emergence of Speech Emotion Recognition (SER) as a focal point of research into speech processing reflects its significance in the field of Human-Computer Interaction (HCI). It is core required functionality for a variety of applications and a high degree of accuracy is critical; activities as diverse as evaluating levels of emotion in children in care and measuring customer satisfaction. The extent of demand for accurate SER is reflected in the significant number of papers that have been published and studies performed. An innovative approach to speech recognition is presented in this paper centered on a cloud model alongside the conventional system for measuring emotion in speech. There are multiple stages involved in detecting and identifying emotions in speech from audio clips. The initial pre-processing stage detects the speech in the audio file and applies noise reduction. Next, the system uses Mel-frequency cepstral coefficient (MFCC) algorithms to extract features.This process results in the creation of testing and training datasets populated with the emotions: Neutral, Happiness, Sadness, Fear, Surprise, Disgust and Anger. The classification stage utilizes Support Vector Machine (SVM) classifiers to identify the emotion. An additional step implements a Confusion Matrix (CM) method to assess how these classifiers performed. Testing was executed against RAVDESS and SAVEE databases, where the detection rate achieved against the RAVDESS database was 95.3%.

In the field of machine learning, it is still a critical issue to identify and supervise the learned representation without manually intervening or intuition assistance to extract useful knowledge or serve for the downstream tasks. In this work, we focus on supervising the influential factors extracted by the variational autoencoder (VAE). The VAE is proposed to learn independent low dimension representation while facing the problem that sometimes pre-set factors are ignored. We argue that the mutual information of the input and each learned factor of the representation plays a necessary indicator of discovering the influential factors. We find the VAE objective inclines to induce mutual information sparsity in factor dimension over the data intrinsic dimension and therefore result in some non-influential factors whose function on data reconstruction could be ignored. We show mutual information also influences the lower bound of VAE’s reconstruction error and downstream classification task. To make such indicator applicable, we design an algorithm for calculating the mutual information for VAE and prove its consistency. Experimental results on MNIST, CelebA and DEAP datasets show that mutual information can help determine influential factors, of which some are interpretable and can be used to further generation and classification tasks, and help discover the variant that connects with emotion on DEAP dataset.

In this report first, the threat analysis for the IoRL system and the specific use cases it is applied to are assessed. Based on the obtained results an Integrated Security Framework (ISF) is introduced as a countermeasure and its architecture is described in details.

With the rapid development of augmented reality (AR) and virtual reality (VR) technology, human-computer interaction (HCI) has been greatly improved for gaming interaction of AR and VR control. The finger micro-gesture is one of the important interactive methods for HCI applications such as in the Google Soli and Microsoft Kinect projects. However, the progress in this research is slow due to the lack of high quality public available database. In this paper, holoscopic 3D camera is used to capture high quality micro-gesture images and a new unique holoscopic 3D micro-gesture (HoMG) database is produced. The principle of the holoscopic 3D camera is based on the fly viewing system to see the objects. HoMG database recorded the image sequence of 3 conventional gestures from 40 participants under different settings and conditions. For the purpose of micro-gesture recognition, HoMG has a video subset with 960 videos and a still image subset with 30635 images. Initial micro-gesture recognition on both subsets has been conducted using traditional 2D image and video features and popular classifiers and some encouraging performance has been achieved. The database will be available for the research communities and speed up the research in this area., 6 pages, 4 figures

This report describes the system hardware and software architecture of the radio light receiver. The software architecture and building blocks of layer1, layer 2 and layer 3 of the radio light receiver including the intra handover are also described in this report. This report provides also the details of the radio light receiver interface of both mmW Tx/Rx module and VLC receiver. This report also describes the details of the protocols for the IoRL positioning solution. Finally, the consumer products’ design and interfaces are presented in this report.

This report describes the system hardware architecture of the remote radio light head and UE to support IoRL software structure for Access Layer, Network Layer, Orchestration Layer and Service Layer. The software architecture of Infrastructure Layer, Network Layer, Orchestration Layer and Service Layer is also described in this report. This report provides also the details of IoRL position sensing architecture. Finally, the light system electro-mechanical architecture and consumer products electro-mechanical architecture is presented in this report.

Social touch is an important form of social interaction. In Human Robot Interaction (HRI), touch can provide additional information to other modalities, such as audio, visual. One of the application is the robot therapy that has great social significance. In this paper, an ensemble classifier based on threeway decisions is proposed to recognize touch gestures. Firstly, features are extracted from on six perspectives and four classifiers are constructed on different scales with different pre-processing methods. . Then an ensemble classifier is used to combine the four classifiers to classify the gestures. The proposed method is tested on the public Corpus of Social Touch (Cost) dataset. The experiments results not only verify the validity of our method but also show the better accuracy of our ensemble classifier.

Meaningful facial parts can convey key cues for both facial action unit detection and expression prediction. Textured 3D face scan can provide both detailed 3D geometric shape and 2D texture appearance cues of the face which are beneficial for Facial Expression Recognition (FER). However, accurate facial parts extraction as well as their fusion are challenging tasks. In this paper, a novel system for 3D FER is designed based on accurate facial parts extraction and deep feature fusion of facial parts. In particular, each textured 3D face scan is firstly represented as a 2D texture map and a depth map with one-to-one dense correspondence. Then, the facial parts of both texture map and depth map are extracted using a novel 4-stage process consists of facial landmark localization, facial rotation correction, facial resizing, facial parts bounding box extraction and post-processing procedures. Finally, deep fusion Convolutional Neural Networks (CNNs) features of all facial parts are learned from both texture maps and depth maps, respectively and nonlinear SVMs are used for expression prediction. Experiments are conducted on the BU-3DFE database, demonstrating the effectiveness of combing different facial parts, texture and depth cues and reporting the state-of-the-art results in comparison with all existing methods under the same setting., Comment: 7 pages, 3 figures

Micro-expression is a subtle and involuntary facial expression that may reveal the hidden emotion of human beings. Spotting micro-expression means to locate the moment when the microexpression happens, which is a primary step for micro-expression recognition. Previous work in microexpression expression spotting focus on spotting micro-expression from short video, and with hand-crafted features. In this paper, we present a methodology for spotting micro-expression from long videos. Specifically, a new convolutional neural network named as SMEConvNet (Spotting Micro-Expression Convolutional Network) was designed for extracting features from video clips, which is the first time that deep learning is used in micro-expression spotting. Then a feature matrix processing method was proposed for spotting the apex frame from long video, which uses a sliding window and takes the characteristics of micro-expression into account to search the apex frame. Experimental results demonstrate that the proposed method can achieve better performance than existing state-of-art methods.

Recently, real-time facial expression recognition has attracted more and more research. In this study, an automatic facial expression real-time system was built and tested. Firstly, the system and model were designed and tested on a MATLAB environment followed by a MATLAB Simulink environment that is capable of recognizing continuous facial expressions in real-time with a rate of 1 frame per second and that is implemented on a desktop PC. They have been evaluated in a public dataset, and the experimental results were promising. The dataset and labels used in this study were made from videos, which were recorded twice from five participants while watching a video. Secondly, in order to implement in real-time at a faster frame rate, the facial expression recognition system was built on the field-programmable gate array (FPGA). The camera sensor used in this work was a Digilent VmodCAM — stereo camera module. The model was built on the AtlysTM Spartan-6 FPGA development board. It can continuously perform emotional state recognition in real-time at a frame rate of 30. A graphical user interface was designed to display the participant’s video in real-time and two-dimensional predict labels of the emotion at the same time.

In this paper we present a 5G Internet Radio-Light (IoRL) architecture for supermarkets that can be readily deployed because it utilizes unlicensed visible light and millimeter wave part of the spectrum and which is used to provide shoppers with accurate location, interaction, access to Internet and Cloud based services such as high resolution video on a Tablet PC. The paper describes the supermarket use cases, the user and functional requirements and the IoRL architecture.

Automated human emotion detection is a topic of significant interest in the field of computer vision. Over the past decade, much emphasis has been on using facial expression recognition (FER) to extract emotion from facial expressions. In this paper, the proposed system presents a novel method of facial recognition based on the cloud model, in combination with the traditional facial expression system. The process of predicting emotions from facial expression images contains several stages. The first stage of this system is the pre-processing stage, which is applied by detecting the face in images and then resizing the images. The second stage involves extracting features from facial expression images using Facial Landmarks and Center of Gravity (COG) feature extraction algorithms, which generate the training and testing datasets that contain the expressions of Anger, Disgust, Fear, Happiness, Neutrality, Sadness, and Surprise. Support Vector Machine (SVM) classifiers are then used for the classification stage in order to predict the emotion. In addition, a Confusion Matrix (CM) technique is used to evaluate the performance of these classifiers. The proposed system is tested on CK+, JAFFE, and KDEF databases. However, the proposed system achieved a prediction rate of 96.3% when Facial Landmarks and the Center of Gravity (COG)+SVM method are used.

Automated facial expression recognition (AFER) is a crucial technology to and a challenging task for human computer interaction. Previous methods of AFER have incorporated different features and classification methods and use basic testing approaches. In this paper, we employ the best feature descriptor for AFER by empirically evaluating the feature descriptors named the Facial Landmarks descriptor and the Center of Gravity descriptor. We examine each feature descriptor by considering one classification method, such as the Support Vector Machine (SVM) method, with three unique facial expression recognition (FER) datasets. In addition to test accuracies, we present confusion matrices of AFER. We also analyze the effect of using these feature and image resolutions on AFER performance. Our study indicates that the Facial Landmarks descriptor is the best choice to run AFER on mobile phones. The results of our study demonstrate that the proposed facial expression recognition on a mobile phone application is successful and provides up to 96.3% recognition accuracy.

From the last decade, researches on human facial emotion recognition disclosed that computing models built on regression modelling can produce applicable performance. However, many systems need extensive computing power to be run that prevents its wide applications such as robots and smart devices. In this proposed system, a real-time automatic facial expression system was designed, implemented and tested on an embedded device such as FPGA that can be a first step for a specific facial expression recognition chip for a social robot. The system was built and simulated in MATLAB and then was built on FPGA and it can carry out real time continuously emotional state recognition at 30 fps with 47.44% accuracy. The proposed graphic user interface is able to display the participant video and two dimensional predict labels of the emotion in real time together.

Automatic continuous affect recognition from multiple modalities is one of the most active research areas in affective computing. In addressing this regression problem, the advantages of a model, such as Support Vector Regression (SVR), or a model that can capture temporal dependencies within a predefined time window, such as Time Delay Neural Network (TDNN), Long Short-Term Memory (LSTM) or Kalman Filter (KF), have been frequently explored, but in an isolated way. The motivation is towards decoupling temporal information from its features at the semantic level, in order to exploit the slow-changing emotional property at decision level. This paper explores and proposes 2-stage regression framework where SVR, that has been regarded as the baseline approach on affective recognition task, is concatenated together with subsequent models. Extensive experiments have been carried out on a naturalistic emotion dataset, using eight modalities present in RECOLA database. The results shows the proposed framework can capture temporal information at the prediction level, and outperform state-of-theart approaches in continuous affective recognition.

In this paper, an automatic emotion detection system is built for a computer or machine to detect the emotional state from facial expressions in human computer communication. Firstly, dynamic motion features are extracted from facial expression videos and then advanced machine learning methods for classification and regression are used to predict the emotional states. The system is evaluated on two publicly available datasets, i.e. GEMEP_FERA and AVEC2013, and satisfied performances are achieved in comparison with the baseline results provided. With this emotional state detection capability, a machine can read the facial expression of its user automatically. This technique can be integrated into applications such as smart robots, interactive games and smart surveillance systems.

Early detection and diagnosis of Parkinson disease will provide a good chance for patients to take early actions and prevent its further development. In this paper, a smart tool for the diagnosis of Parkinsonian syndromes is designed and developed using low–cost Texas Instruments eZ430-Chronos wireless watches. With this smart tool, Parkinson Bradykinesia is detected based on the cycle of a human gait, with the watch worn on the foot, and Parkinson Tremor shaking is detected and differed by frequency 0 to 8 Hz on the arm in real-time with a developed statistical diagnosis chart. It can be used in small clinics as well as home environment due to its low-cost and easy-use property.

Music makes our life lovely because it can affect our mental states significantly with its emotional information inside. Different people might be affected differently from the same music when they listening the music in different situation and mental states. However, the common emotion information the music can be agreed even from peoples with quite different background and cultures. An automatic emotion recognition system be proposed for the music by extracting different features from the music and machine learning method learning from common knowledge on emotional state of the trained data in this paper. Firstly, two-channel audio signals are processed, and typical audio features are extracted. Then some other features used for EEG signal analysis are also extracted. Finally, these features are combined and the random forest classifier is used for the classification. The proposed method has been used on a public music dataset for test and the experimental results demonstrate its efficiency when it compare with the state-of-the-art performance in the same dataset.

Human brain behavior is very complex and it is difficult to interpret. Human emotion might come from brain activities. However, the relationship between human emotion and brain activities is far from clear. In recent years, more and more researchers are trying to discover this relationship by recording brain signals such as electroencephalogram (EEG) signals with the associated emotion information extracted from other modalities such as facial expression. In this paper, machine learning based methods are used to model this relationship in the publicly available dataset DEAP (Database for Emotional Analysis using Physiological Signals). Different features are extracted from raw EEG recordings. Then Maximum Relevance Minimum Redundancy (mRMR) was used for feature selection. These features are fed into machine learning methods to build the prediction models to extract the emotion information from EEG signals. The models are evaluated on this dataset and satisfactory results are achieved.

Chronic pain is a disease that the patients suffers a lot in their daily life and it is difficult to be released completely. It is difficult to manage because pain can come anytime and it is unpredictable. However, the pain can be represented by the pain related behaviors such as guiding and abrupt actions. In this paper, we will develop a machine learning based system that can detect the pain related behaviors from patient’s Electromyography (EMG) signals and body movements continuously. The system includes data collection, feature extraction, modeling and classification. The data were collected using biosensor sensor for EMG and motion capture for body movement. Specific features are extracted from the body movement data. Then Random Forest and a Two Stage Classification (TSC) scheme (KNN coupled with Hidden Markov Model (HMM)) were used for pain related behavior detection in a continuous manner. The proposed method was tested on Emo-Pain corpus dataset provided by UCL and experimental results demonstrate the efficiency of the proposed method.

The smartphone has become an indispensable part in people’s life. Identifying the user's emotional state according to the usage of smartphone is a new way to improve the human- computer interaction and user experience. In this paper, we present an attempt to recognize emotional states by using finger- stroke pattern. Firstly, International Affective Picture System (IAPS) were used to design the emotion inducing experiment. Then finger-stroke features under different emotional categories were extracted and analyzed. Ultimately, we used machine learning algorithms to identify three basic emotional states including positive, neutral, and negative. The experiment results show that the stroke exists some specific behavioral patterns between different people. For all 24 subjects, the average classification accuracy rate reached 85.1%, and we got the average recognition accuracy at 72.3%, 74.6% and 69.6% for male, female and all subjects. This work is supported by ChongQing postgraduate scientific research and innovation projects of China, partly funded by the National Nature Science Foundation of China (61379114) and the Natural Science Foundation of Chongqing (CSTC2014jcyjA40047) and the Doctoral startup project of CQUPT (A2014-20).

The acknowledgement text of the initially published paper was incomplete. It should have been as follows

As the overall population ages, driving-related accidents and injuries, associated with elderly drivers, have risen. Existing research about elderly drivers mainly focuses on factual data collection and analysis, indicating the elderly’s growing fatal accident rates and their different behaviours compared to younger drivers. However, few research has focused on design-led practical solutions to mitigate the elderly’s growing fatal accidents, by considering their usability and body conditions, afflicting the elderly, such as decreased vision, hearing, and reaction times. In this paper, first, current worldwide situations on growing fatal accident rates for elderly drivers is reviewed and the key impact factors are identified and discussed with regarding to usability and design trend in the automotive technology for elderly. Second, existing smart vehicle technology-based solutions to promote safe driving are explored and their pros and cons are discussed and analysed. Most of solutions are not created by people with driving difficulties, which are caused by health problems most commonly afflicting the elderly. Thirdly, diverse design-led research activities are taken, such as a survey, observation, and interviews to gain new understanding of what kinds of driving problems elderly drivers have and demonstrate how new system concepts could be developed for the elderly’s benefits. Finally, it is found that the elderly’s low vision and late reaction are main factors causing their driving difficulties. Based on this finding, usable vehicle system design ideas have been proposed, by utilising facial expression sensing technology as a solution. The proposed solutions would ensure reducing both the elderly’s driving problems and high fatal accident rates and provide a more enjoyable driving environment for the elderly population.

Touch is a primary nonverbal communication channel used to communicate emotions or other social messages. Despite its importance, this channel is still very little explored in the affective computing field, as much more focus has been placed on visual and aural channels. In this paper, we investigate the possibility to automatically discriminate between different social touch types. We propose five distinct feature sets for describing touch behaviours captured by a grid of pressure sensors. These features are then combined together by using the Random Forest and Boosting methods for categorizing the touch gesture type. The proposed methods were evaluated on both the HAART (7 gesture types over different surfaces) and the CoST (14 gesture types over the same surface) datasets made available by the Social Touch Gesture Challenge 2015. Well above chance level performances were achieved with a 67% accuracy for the HAART and 59% for the CoST testing datasets respectively.

Copyright @ Elsevier Inc. All rights reserved. Bio-inspired vision sensors are particularly appropriate candidates for navigation of vehicles or mobile robots due to their computational simplicity, allowing compact hardware implementations with low power dissipation. The Lobula Giant Movement Detector (LGMD) is a wide-field visual neuron located in the Lobula layer of the Locust nervous system. The LGMD increases its firing rate in response to both the velocity of an approaching object and the proximity of this object. It has been found that it can respond to looming stimuli very quickly and trigger avoidance reactions. It has been successfully applied in visual collision avoidance systems for vehicles and robots. This paper introduces a modified neural model for LGMD that provides additional depth direction information for the movement. The proposed model retains the simplicity of the previous model by adding only a few new cells. It has been simplified and implemented on a Field Programmable Gate Array (FPGA), taking advantage of the inherent parallelism exhibited by the LGMD, and tested on real-time video streams. Experimental results demonstrate the effectiveness as a fast motion detector. UK Trade and Strategy Board under Grant TP/2/SC/6/1/10444.