Search

Permanent pacemaker (PPM) implantation occurs in up to 5 % of patients after cardiac surgery but there is little consensus on how long to wait between surgery and PPM insertion. Predicting the likelihood of a patient being pacing dependent 30 days after implant can aid with this timing decision and avoid unnecessary observation time waiting for intrinsic conduction to recover. In this paper, we introduce a new approach for the prediction of PPM dependency at 30 days after implant in patients who have undergone recent cardiac surgery. The aim is to create an automatic detection model able to support clinicians in the decision-making process. We first applied Synthetic Minority Oversampling Technique (SMOTE) and Bayesian Networks (BN) to the dataset, to balance the inherently imbalanced data and create additional synthetic data respectively. The six resultant datasets were then used to train four different classifiers to predict pacing dependence at 30 days, all using the same testing set. The Bagged Trees classifier achieved the best results, reaching an area under the receiver operating curve (AUC) of 90 % in the train phase, and 83 % in the test phase. The overall classification performance was clearly enhanced when using SMOTE and synthetic data created with BN to create a combined and balanced dataset. This technique could be of great use in answering clinical questions where the original dataset is imbalanced.

The popular deep convolutional networks used for image super-resolution (SR) reconstruction often increase the network depth and employ attention mechanism to improve image reconstruction effect. However, these networks suffer from two problems. The first is the deeper network easily causes higher computational cost and more GPU memory usage. The second is traditional attention mechanism often misses the spatial information of images leading the loss of image detail information. To address these issues, we propose a lightweight non-local network (LNLN) for image super resolution in this paper. The proposed network makes two contributions. First, we use non-local module instead of normal attention module to obtain larger receptive field and extract more comprehensive feature information, which is helpful for improving image SR reconstruction results. Secondly, we use the depthwise separable convolution (DSC) instead of the vanilla convolution to reconstruct the residual block, which greatly reduces the number of parameters and computational cost. The proposed LNLN and comparative networks are evaluated on five commonly public datasets, and experiments demonstrate that the proposed LNLN is superior to state-of-the-art networks in terms of reconstruction performance, the number of parameters and storage space.

Since signal-to-noise ratio and axial resolution are effective decision parameters in ultrasound imaging and ultrasonic testing system, we combined nonlinear frequency modulation and Barker code sequence to improve these parameters, thereby improving the effect and performance in the ultrasound imaging and ultrasonic testing system. The theoretical study on the signal generation and decoding has been presented and the cyst phantom simulation has been carried out. The theoretical analysis shows that the new code method can improve 8.46 dB in Contrast ratio compared to simple pulse signal. The simulation produced the similar results.

The V-Net based 3D fully convolutional neural networks have been widely used in liver volumetric data segmentation. However, due to the large number of parameters of these networks, 3D FCNs suffer from high computational cost and GPU memory usage. To address these issues, we design a lightweight V-Net (LV-Net) for liver segmentation in this paper. The proposed network makes two contributions. The first is that we design an inverted residual bottleneck block (IRB block) and a 3D average pooling block and apply them to the proposed LV-Net. Compared with vanilla convolution, depth-wise convolution and point-wise convolution employed by the IRB block can not only reduce the number of parameters significantly, but also extract features sufficiently well by decoupling cross-channel corrections and spatial correlations. The second is that the LV-Net employs 3D deep supervision to improve the final loss function in training phase, which makes the proposed LV-Net acquire a more powerful discrimination capability between liver areas and non-liver areas. The proposed LV-Net is evaluated on public LiTS dataset, and experiments demonstrate that the proposed LV-Net is superior to popular 2D and 3D networks in terms of segmentation performance, parameter quantity and computational cost.

Face recognition has made significant progress in recent years. It has been successfully applied to many applications from security entrance systems for staff identification to visual surveillance. However, the performance of current systems are highly affected by factors such as subject’s head orientation, facial expressions, and face wear-on. In order to address this challenge, researchers usually resort to more training data or more complex recognition models. We believe the secret lies in using a more rich data source. Therefore, in this paper, we use a holoscopic 3D (H3D) imaging system. To show the effectiveness of the sensor, we use it with a simple back propagation neural network (BPNN) classifier. We illustrate the accuracy of the system through an experiment and show that it has high accuracy and is robust against challenges aforementioned.

With the rapid development of deep learning (DL) in recent years, automatic modulation recognition (AMR) with DL has achieved high accuracy. However, aiming to obtain higher classification accuracy, DL requires numerous training samples. In order to solve this problem, it is a challenge to study how to efficiently use DL for AMR in the case of few samples. In this letter, inspired by the capsule network (CapsNet), we propose a new network structure named AMR-CapsNet to achieve higher classification accuracy of modulation signals with fewer samples, and further analyze the adaptability of DL models in the case of few samples. The simulation results demonstrate that when 3% of the dataset is used to train and the signal-to-noise ratio (SNR) is greater than 2 dB, the overall classification accuracy of the AMR-CapsNet is greater than 80%. Compared with convolutional neural network (CNN), the classification accuracy is improved by 20%.

Finger microgestures have been widely used in human computer interaction (HCI), particularly for interactive applications, such as virtual reality (VR) and augmented reality (AR) technologies, to provide immersive experience. However, traditional 2D image-based microgesture recognition suffers from low accuracy due to the limitations of 2D imaging sensors, which have no depth information. In this article, we proposed an innovative 3D microgesture recognition system based on a holoscopic 3D imaging sensor. Due to the lack of holoscopic 3D datasets, a comprehensive holoscopic 3D microgesture (HoMG) database is created and used to develop a robust 3D microgesture recognition method. Then, a fast algorithm is proposed to extract multiviewpoint images from one holoscopic image. Furthermore, we applied a CNN model with an attention-based residual block to each viewpoint image to improve the algorithm performance. Finally, bagging classification tree decision-level fusion is applied to combine the predictions. The experimental results demonstrate that the proposed method outperforms state-of-the-art methods and delivers a better accuracy than existing methods.

In this paper, we propose a novel approach based on a symmetric fully convolutional network within pyramid pooling (FCN-PP) for landslide mapping (LM). The proposed approach has three advantages. Firstly, this approach is automatic and insensitive to noise because multivariate morphological reconstruction (MMR) is used for image preprocessing. Secondly, it is able to take into account features from multiple convolutional layers and explore efficiently the context of images, which leads to a good tradeoff between wider receptive field and the use of context. Finally, the selected pyramid pooling module addresses the drawback of single-scale pooling employed by convolutional neural network (CNN), fully convolutional network (FCN), U-Net, etc. Experimental results show that the proposed FCN-PP is effective for LM, and it outperforms state-of-the-art approaches in terms of four metrics, Precision, Recall, F -score, and Accuracy.

Because emotions are such a significant and integral part of being human, understanding them and knowing how to react to the emotions of others is a fundamental requirement for successful social interaction. We recognize emotions primarily through speech and facial expression. The topic is gaining importance in academic research triggered by research in new techniques such as identifying emotions based on speech context. This investigates the relationship between emotions and the content of our speech. This paper proposes how emotion in speech and facial expression can be recognized in real time using a framework consisting of mobile phone technology backed by cloud computing. This functionality was developed and built into a mobile phone application. Currently, the application works on any Android smartphone to detect and identify emotions in real time. The results are expressed as a percentage of all the possible emotions, such as sad, happy, fear, surprise, anger and so on. The results of the experiment confirm that face and speech emotion recognition was conducted successfully using a smartphone. It was correct in 97.26% of instances when used with standard corpora: a.Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS) b.the Surrey Audio-Visual Expressed Emotion (SAVEE)

The emergence of Speech Emotion Recognition (SER) as a focal point of research into speech processing reflects its significance in the field of Human-Computer Interaction (HCI). It is core required functionality for a variety of applications and a high degree of accuracy is critical; activities as diverse as evaluating levels of emotion in children in care and measuring customer satisfaction. The extent of demand for accurate SER is reflected in the significant number of papers that have been published and studies performed. An innovative approach to speech recognition is presented in this paper centered on a cloud model alongside the conventional system for measuring emotion in speech. There are multiple stages involved in detecting and identifying emotions in speech from audio clips. The initial pre-processing stage detects the speech in the audio file and applies noise reduction. Next, the system uses Mel-frequency cepstral coefficient (MFCC) algorithms to extract features.This process results in the creation of testing and training datasets populated with the emotions: Neutral, Happiness, Sadness, Fear, Surprise, Disgust and Anger. The classification stage utilizes Support Vector Machine (SVM) classifiers to identify the emotion. An additional step implements a Confusion Matrix (CM) method to assess how these classifiers performed. Testing was executed against RAVDESS and SAVEE databases, where the detection rate achieved against the RAVDESS database was 95.3%.

Automatic continuous affect recognition from multiple modality in the wild is arguably one of the most challenging research areas in affective computing. In addressing this regression problem, the advantages of the each modality, such as audio, video and text, have been frequently explored but in an isolated way. Little attention has been paid so far to quantify the relationship within these modalities. Motivated to leverage the individual advantages of each modality, this study investigates behavioral modeling of continuous affect estimation, in multimodal fusion approaches, using Linear Regression, Exponent Weighted Decision Fusion and Multi-Gene Genetic Programming. The capabilities of each fusion approach are illustrated by applying it to the formulation of affect estimation generated from multiple modality using classical Support Vector Regression. The proposed fusion methods were applied in the public Sentiment Analysis in the Wild (SEWA) multimodal dataset and the experimental results indicate that employing proper fusion can deliver a significant performance improvement for all affect estimation. The results further show that the proposed systems is competitive or outperform the other state-of-the-art approaches.

With the rapid development of augmented reality (AR) and virtual reality (VR) technology, human-computer interaction (HCI) has been greatly improved for gaming interaction of AR and VR control. The finger micro-gesture is one of the important interactive methods for HCI applications such as in the Google Soli and Microsoft Kinect projects. However, the progress in this research is slow due to the lack of high quality public available database. In this paper, holoscopic 3D camera is used to capture high quality micro-gesture images and a new unique holoscopic 3D micro-gesture (HoMG) database is produced. The principle of the holoscopic 3D camera is based on the fly viewing system to see the objects. HoMG database recorded the image sequence of 3 conventional gestures from 40 participants under different settings and conditions. For the purpose of micro-gesture recognition, HoMG has a video subset with 960 videos and a still image subset with 30635 images. Initial micro-gesture recognition on both subsets has been conducted using traditional 2D image and video features and popular classifiers and some encouraging performance has been achieved. The database will be available for the research communities and speed up the research in this area., 6 pages, 4 figures

Automated human emotion detection is a topic of significant interest in the field of computer vision. Over the past decade, much emphasis has been on using facial expression recognition (FER) to extract emotion from facial expressions. In this paper, the proposed system presents a novel method of facial recognition based on the cloud model, in combination with the traditional facial expression system. The process of predicting emotions from facial expression images contains several stages. The first stage of this system is the pre-processing stage, which is applied by detecting the face in images and then resizing the images. The second stage involves extracting features from facial expression images using Facial Landmarks and Center of Gravity (COG) feature extraction algorithms, which generate the training and testing datasets that contain the expressions of Anger, Disgust, Fear, Happiness, Neutrality, Sadness, and Surprise. Support Vector Machine (SVM) classifiers are then used for the classification stage in order to predict the emotion. In addition, a Confusion Matrix (CM) technique is used to evaluate the performance of these classifiers. The proposed system is tested on CK+, JAFFE, and KDEF databases. However, the proposed system achieved a prediction rate of 96.3% when Facial Landmarks and the Center of Gravity (COG)+SVM method are used.

Automated facial expression recognition (AFER) is a crucial technology to and a challenging task for human computer interaction. Previous methods of AFER have incorporated different features and classification methods and use basic testing approaches. In this paper, we employ the best feature descriptor for AFER by empirically evaluating the feature descriptors named the Facial Landmarks descriptor and the Center of Gravity descriptor. We examine each feature descriptor by considering one classification method, such as the Support Vector Machine (SVM) method, with three unique facial expression recognition (FER) datasets. In addition to test accuracies, we present confusion matrices of AFER. We also analyze the effect of using these feature and image resolutions on AFER performance. Our study indicates that the Facial Landmarks descriptor is the best choice to run AFER on mobile phones. The results of our study demonstrate that the proposed facial expression recognition on a mobile phone application is successful and provides up to 96.3% recognition accuracy.

From the last decade, researches on human facial emotion recognition disclosed that computing models built on regression modelling can produce applicable performance. However, many systems need extensive computing power to be run that prevents its wide applications such as robots and smart devices. In this proposed system, a real-time automatic facial expression system was designed, implemented and tested on an embedded device such as FPGA that can be a first step for a specific facial expression recognition chip for a social robot. The system was built and simulated in MATLAB and then was built on FPGA and it can carry out real time continuously emotional state recognition at 30 fps with 47.44% accuracy. The proposed graphic user interface is able to display the participant video and two dimensional predict labels of the emotion in real time together.

Automatic continuous affect recognition from multiple modalities is one of the most active research areas in affective computing. In addressing this regression problem, the advantages of a model, such as Support Vector Regression (SVR), or a model that can capture temporal dependencies within a predefined time window, such as Time Delay Neural Network (TDNN), Long Short-Term Memory (LSTM) or Kalman Filter (KF), have been frequently explored, but in an isolated way. The motivation is towards decoupling temporal information from its features at the semantic level, in order to exploit the slow-changing emotional property at decision level. This paper explores and proposes 2-stage regression framework where SVR, that has been regarded as the baseline approach on affective recognition task, is concatenated together with subsequent models. Extensive experiments have been carried out on a naturalistic emotion dataset, using eight modalities present in RECOLA database. The results shows the proposed framework can capture temporal information at the prediction level, and outperform state-of-theart approaches in continuous affective recognition.

Music makes our life lovely because it can affect our mental states significantly with its emotional information inside. Different people might be affected differently from the same music when they listening the music in different situation and mental states. However, the common emotion information the music can be agreed even from peoples with quite different background and cultures. An automatic emotion recognition system be proposed for the music by extracting different features from the music and machine learning method learning from common knowledge on emotional state of the trained data in this paper. Firstly, two-channel audio signals are processed, and typical audio features are extracted. Then some other features used for EEG signal analysis are also extracted. Finally, these features are combined and the random forest classifier is used for the classification. The proposed method has been used on a public music dataset for test and the experimental results demonstrate its efficiency when it compare with the state-of-the-art performance in the same dataset.

Human brain behavior is very complex and it is difficult to interpret. Human emotion might come from brain activities. However, the relationship between human emotion and brain activities is far from clear. In recent years, more and more researchers are trying to discover this relationship by recording brain signals such as electroencephalogram (EEG) signals with the associated emotion information extracted from other modalities such as facial expression. In this paper, machine learning based methods are used to model this relationship in the publicly available dataset DEAP (Database for Emotional Analysis using Physiological Signals). Different features are extracted from raw EEG recordings. Then Maximum Relevance Minimum Redundancy (mRMR) was used for feature selection. These features are fed into machine learning methods to build the prediction models to extract the emotion information from EEG signals. The models are evaluated on this dataset and satisfactory results are achieved.

Chronic pain is a disease that the patients suffers a lot in their daily life and it is difficult to be released completely. It is difficult to manage because pain can come anytime and it is unpredictable. However, the pain can be represented by the pain related behaviors such as guiding and abrupt actions. In this paper, we will develop a machine learning based system that can detect the pain related behaviors from patient’s Electromyography (EMG) signals and body movements continuously. The system includes data collection, feature extraction, modeling and classification. The data were collected using biosensor sensor for EMG and motion capture for body movement. Specific features are extracted from the body movement data. Then Random Forest and a Two Stage Classification (TSC) scheme (KNN coupled with Hidden Markov Model (HMM)) were used for pain related behavior detection in a continuous manner. The proposed method was tested on Emo-Pain corpus dataset provided by UCL and experimental results demonstrate the efficiency of the proposed method.

3D facial expression recognition has gained more and more interests from affective computing society due to issues such as pose variations and illumination changes caused by 2D imaging having been eliminated. There are many applications that can benefit from this research, such as medical applications involving the detection of pain and psychological effects in patients, in human-computer interaction tasks that intelligent systems use in today's world. In this paper, we look into 3D Facial Expression Recognition, by investigating many feature extraction methods used on the 2D textured images and 3D geometric data, fusing the 2 domains to increase the overall performance. A One Vs All Multi-class SVM Classifier has been adopted to recognize the expressions Angry, Disgust, Fear, Happy, Neutral, Sad and Surprise from the BU-3DFE and Bosphorus databases. The proposed approach displays an increase in performance when the features are fused together.

Automatic affective dimension recognition from facial expression continuously in naturalistic contexts is a very challenging research topic but very important in human-computer interaction. In this paper, an automatic recognition system was proposed to predict the affective dimensions such as Arousal, Valence and Dominance continuously in naturalistic facial expression videos. Firstly, visual and vocal features are extracted from image frames and audio segments in facial expression videos. Secondly, a wavelet transform based digital filtering method is applied to remove the irrelevant noise information in the feature space. Thirdly, Partial Least Squares regression is used to predict the affective dimensions from both video and audio modalities. Finally, two modalities are combined to boost overall performance in the decision fusion process. The proposed method is tested in the fourth international Audio/Visual Emotion Recognition Challenge (AVEC2014) dataset and compared to other state-of-the-art methods in the affect recognition sub-challenge with a good performance.

Automatic detection and analysis of the emotional states provide important clues for predicting personal behaviors. The techniques are essential for monitoring and protecting such vulnerable people as patients with mental health conditions, persons under heavy stress, and young kids with neurodevelopmental disorders. However, few automatic emotional state recognition systems have been put into real-world applications. Among various reasons leading the lack of success, the overwhelming computing complexity is a major hurdle. In this paper, a continuously emotional state monitoring system that is capable of recognizing naturalistic facial expression in realtime is designed and implemented on a desktop PC with GPU as an accelerator. It has been tested in real scenarios and the experimental results prove promising.

This is the post-print version of the Article. The official published version can be accessed from the link below - Copyright @ 2012 IEEE This paper introduces a tri-state logic self-organizing map (bSOM) designed and implemented on a field programmable gate array (FPGA) chip. The bSOM takes binary inputs and maintains tri-state weights. A novel training rule is presented. The bSOM is well suited to FPGA implementation, trains quicker than the original self-organizing map (SOM), and can be used in clustering and classification problems with binary input data. Two practical applications, character recognition and appearance-based object identification, are used to illustrate the performance of the implementation. The appearance-based object identification forms part of an end-to-end surveillance system implemented wholly on FPGA. In both applications, binary signatures extracted from the objects are processed by the bSOM. The system performance is compared with a traditional SOM with real-valued weights and a strictly binary weighted SOM.

In this paper, a Naive Bayes classifier was simplified and implemented as a multi-class classifier for binary feature vectors. It was designed on FPGA using very limited hardware resources and runs quickly and efficiently in both training and testing phases. It was first tested on a handwriting digital number dataset, and then applied in the visual object recognition on a single FPGA based visual surveillance system. It was compared with a binary Self Organizing Map (bSOM) using tri-states operation on FPGA, and the experimental results demonstrated both its higher performance and lower resource usage on the FPGA chip.

This paper introduces the implementation of an FPGA-based tri-state rule binary Self Organizing Map (bSOM), which takes binary inputs and maintains tri-state weights, with a node labelling algorithm which makes it capable of object classification. The bSOM is used for appearance-based object identification during tracking in video sequences. It is designed to provide part of an end-to-end surveillance system implemented wholly on FPGA. It is trained off-line using a labelled training data set for nine objects, using binary signatures extracted from the colour histogram, and successfully used for appearance-based identification of objects in approximately 85% of cases in a fairly challenging data set. The paper identifies how this preliminary work can be extended to provide full on-line appearance-based identification and tracking.

A binary Self Organizing Map (SOM) has been designed and implemented on a Field Programmable Gate Array (FPGA) chip. A novel learning algorithm which takes binary inputs and maintains tri-state weights is presented. The binary SOM has the capability of recognizing binary input sequences after training. A novel tri-state rule is used in updating the network weights during the training phase. The rule implementation is highly suited to the FPGA architecture, and allows extremely rapid training. This architecture may be used in real-time for fast pattern clustering and classification of binary features.

The Sparse Distributed Memory (SDM) proposed by Kanerva provides a simple model for human long-term memory, with a strong underlying mathematical theory. However, there are problematic features in the original SDM model that affect its efficiency and performance in real world applications and for hardware implementation. In this paper, we propose modifications to the SDM model that improve its efficiency and performance in pattern recall. First, the address matrix is built using training samples rather than random binary sequences. This improves the recall performance significantly. Second, the content matrix is modified using a simple tri-state logic rule. This reduces the storage requirements of the SDM and simplifies the implementation logic, making it suitable for hardware implementation. The modified model has been tested using pattern recall experiments. It is found that the modified model can recall clean patterns very well from noisy inputs.

The Lobula Giant Movement Detector (LGMD) is a wide-field visual neuron that is located in the Lobula layer of the Locust nervous system. The LGMD increases its firing rate in response to both the velocity of the approaching object and its proximity. It has been found that it can respond to looming stimuli very quickly and can trigger avoidance reactions whenever a rapidly approaching object is detected. It has been successfully applied in visual collision avoidance systems for vehicles and robots. This paper proposes a modified LGMD model that provides additional movement depth direction information. The proposed model retains the simplicity of the previous neural network model, adding only a few new cells. It has been tested on both simulated and recorded video data sets. The experimental results shows that the modified model can very efficiently provide stable information on the depth direction of movement.

In this paper, we propose a human action recognition system suitable for embedded computer vision applications in security systems, human-computer interaction and intelligent environments. Our system is suitable for embedded computer vision application based on three reasons. Firstly, the system was based on a linear support vector machine (SVM) classifier where classification progress can be implemented easily and quickly in embedded hardware. Secondly, we use compacted motion features easily obtained from videos. We address the limitations of the well known motion history image (MHI) and propose a new hierarchical motion history histogram (HMHH) feature to represent the motion information. HMHH not only provides rich motion information, but also remains computationally inexpensive. Finally, we combine MHI and HMHH together and extract a low dimension feature vector to be used in the SVM classifiers. Experimental results show that our system achieves significant improvement on the recognition performance.

An ASIC implementation of spatial combinative lifting algorithm (SCLA) based 2D forward and inverse discrete wavelet transform (DWT) with both 5/3 and 9/7 filters and 5-level Mallat decomposition is proposed in this paper. This processor is fabricated in a 1.5mm times 1.5 mm die, using UMC 0.18mum CMOS technology, containing 28K gates plus 47Kbits on-chip SRAM. Test shows this chip can process at 23.29 frames/s with image resolution up to 1920times1080 pixels (YUV422 full color) under 100MHz, and consuming 50mW under a 1.8V power supply

The Discrete Wavelet Transform (DWT) is the basis for many image compression techniques, such as JPEG2000, MPEG-4, etc. The Spatial Combinative Lifting Algorithm (SCLA) based 2-D DWT/IDWT requires fewer computations than the conventional Lifting Based Implementation (LBI). In comparison with the LBI, SCLA with the 9/7 filter has 7/12 the number of multiplications. This paper proposes a novel VLSI architecture to compute multilevel SCLA based 2-D DWT/IDWT with the 9/7 filter. The line based transform is integrated with the SCLA scheme to reduce the hardware cost and achieve higher hardware utilization. Efficient organization of six line-buffer memories is used to address the high memory bandwidth requirements. This architecture can be used as a compact and efficient core for JPEG2000 VLSI implementation and various real-time image/video applications.

We have designed a VLSI chip of spatial combinative lifting algorithm (SCLA) based 2-D biorthogonal DWT/IDWT. This DWT/IDWT processor is implemented with 9/7, 5/3 Daubechies filters and 5-level Mallat decomposition method, which can possess 30 frames per second with image resolution up to 1280 t 1024 t 24 bits under 50 MHz system clock. This processor is fabricated with DONGBU 0.25 mm 1P4M standard CMOS technology, with 25k logic gates plus 93k bits on-chip memory and 1.1mW/MHz power consumption, in a 7.84 mm2 die size, which can be used as a compact and efficient hard IP core for JPEG2000 codec VLSI implementation and many other real-time video/audio applications.

It is proposed that the Gabor (1946) expansion coefficients of a periodic (or finite) discrete signal in both the critical sampling case and the oversampling case can be computed by TH neural networks. Theoretical analysis and simulations show that this network will certainly provide approximate solutions arbitrarily close to the accurate one within the time of the order of time constants of this network without any programming complexity.

The performance of the Daubechies (1988) wavelet filters in random signal processing is studied. By using the operator matrices of the compactly supported orthogonal wavelet transform, the quantitative indices of the Daubechies filters in data compression, generalized Wiener filtering and transform coding is obtained. These values can be compared with the other orthogonal transforms.