Search

Electrocardiogram (ECG) signals are a measure minute electrical signals generated during the cardiac cycle, a biometric signal that occurs during vital human activity. ECG signals are susceptible to various types of noise depending on the data acquisition conditions, with factors such as sensor placement and the physiological and mental states of the subject contributing to the diverse shapes of these signals. When the data are acquired in a single session, the environmental variables are relatively similar, resulting in similar ECG signals; however, in subsequent sessions, even for the same person, changes in the environmental variables can alter the signal shape. This phenomenon poses challenges for person identification using ECG signals acquired on different days. To improve the performance of individual identification, even when ECG data is acquired on different days, this paper proposes an ensemble deep neural network for person identification by comparing and analyzing the ECG recognition performance under various conditions. The proposed ensemble deep neural network comprises three streams that incorporate two well-known pretrained models. Each network receives the time-frequency representation of ECG signals as input, and a stream reuses the same network structure under different learning conditions with or without data augmentation. The proposed ensemble deep neural network was validated on the Physikalisch-Technische Bundesanstalt dataset, and the results confirmed a 3.39% improvement in accuracy compared to existing methods.

Current advancements in biosignal-based user recognition technology are paving the way for a next-generation solution that addresses the limitations of face- and fingerprint-based user recognition methods. However, existing biosignal benchmark databases (DBs) for user recognition often suffer from limitations, such as data collection from a small number of subjects in a single session, hindering comprehensive analysis of biosignal variability. This study introduces CSU_MBDB1 and CSU_MBDB2, databases containing electrocardiogram (ECG) and electromyogram (EMG) signals from diverse experimental subjects recorded across multiple sessions. These in-house DBs comprise ECG and EMG data recorded in multiple sessions from 36 and 58 subjects, respectively, with a time interval of more than one day between sessions. During the experiments, subjects performed a total of six gestures while comfortably seated at a desk. CSU_MBDB1 and CSU_MBDB2 consist of three identical gestures, providing expandable data for various applications. When the two DBs are expanded, ECGs and EMGs from 94 subjects can be used, which is the largest number among the multi-biosignal benchmark DBs built by multi-sessions. To assess the usability of the constructed DBs, a user recognition experiment was conducted, resulting in an accuracy of 66.39% for ten subjects. It is important to emphasize that we focused on demonstrating the applicability of the constructed DBs using a basic neural network without signal denoising capabilities. While this approach results in a sacrifice in accuracy, it concurrently provides substantial opportunities for performance enhancement through the implementation of optimized algorithms. Adapting signal denoising processes to the constructed DBs and designing a more sophisticated neural network would undoubtedly contribute to improving the recognition accuracy. Consequently, these constructed DBs hold promise in user recognition, offering valuable research for future investigations. Additionally, DBs can be used in research to analyze the nonlinearity characteristics of ECG and EMG.

When acquiring electrocardiogram (ECG) signals, the placement of electrode patches is crucial for acquiring electrocardiographic signals. Constant displacement positions are essential for ensuring the consistency of the ECG signal when used for individual identification. However, achieving constant placement of ECG electrode patches in every trial for data acquisition is challenging. This is because different individuals may attach patches, and even when the same person attaches them, it may be difficult to specify the exact position. Therefore, gathering ECG data from various locations is necessary. However, this process requires a substantial amount of labor and time, owing to the requirement for multiple attempts. Nonetheless, persisting with these efforts enables the endurance of some ECG differences. To reduce labor and time, we propose a semi-supervised domain adaptation for individual identification using ECG signals. The method operates with a full set of original ECG signals and a small set of ECG signals from different placements to account for the differences between the signals in the generative adversarial network (CycleGAN). Specifically, to train the CycleGAN, the ECG signals were transformed into time–frequency representations, and the trained generator was used to generate ECG signals to expand the small set of ECG signals from different placements. Subsequently, both the original and generated signals were used to train the classifier for individual identification. This scenario can also be applied to the classification of ECG signals from different sensors. The PTB-ECG dataset was used for this experiment. We found that the proposed method showed higher accuracy than when only the original ECG signals were used for the training classifier.

This paper presents a method for recognizing behaviors in videos based on the ensemble RGB-S deep neural network, which combines RGB images and skeleton features from an action recognition database built in intelligent home service robot environments. The ensemble model is designed using the three-stream approach. The first stream classifies behaviors in videos using a convolutional neural network (CNN) based on a pre-trained ResNet101 model, which uses two-dimensional (2D) sequence images of actions as its input, and training a long short-term memory (LSTM) neural network with the sequence (RGB 2D-CNN + LSTM). The second stream directly manages the video and uses a three-dimensional (3D) CNN to include both temporal and spatial information. The 3D CNN is based on a pre-trained R3D-18 model (RGB 3D-CNN). The last stream uses the pose evolution image (PEI) method, which converts the skeleton sequence into a single-color image. The converted images are used as the input for the CNN (Skeleton PEI-2D-CNN). This approach not only reflects the spatial and temporal features of the behaviors in videos, but also includes all characteristics of the 2D sequence images, 3D videos, and skeleton sequences. Finally, a large-scale database for behavior recognition in videos, known as ETRI-Activity3D, is used in this study to verify the performance of the proposed deep neural network. A recognition performance of 93.2% is achieved in a cross-subject experiment, verifying the superiority of this method over models from previous studies.

This paper is concerned with individual identification by late fusion of two-stream deep networks from Electromyogram (EMG) signals. EMG signal has more advantages on security compared to other biosignals exposed visually, such as the face, iris, and fingerprints, when used for biometrics, at least in the aspect of visual exposure, because it is measured through contact without any visual exposure. Thus, we propose an ensemble deep learning model by late information fusion of convolutional neural networks (CNN) and long short-term memory (LSTM) from EMG signals for robust and discriminative biometrics. For this purpose, in the ensemble model’s first stream, one-dimensional EMG signals were converted into time–frequency representation to train a two-dimensional convolutional neural network (EmgCNN). In the second stream, statistical features were extracted from one-dimensional EMG signals to train a long short-term memory (EmgLSTM) that uses sequence input. Here, the EMG signals were divided into fixed lengths, and feature values were calculated for each interval. A late information fusion is performed by the output scores of two deep learning models to obtain a final classification result. To confirm the superiority of the proposed method, we use an EMG database constructed at Chosun University and a public EMG database. The experimental results revealed that the proposed method showed performance improvement by 10.76% on average compared to a single stream and the previous methods.

In modern society, the population has been aging as the lifespan has increased owing to the advancement in medical technologies. This could pose a threat to the economic system and, in serious cases, to the ethics regarding the socially-weak elderly. An analysis of the behavioral characteristics of the elderly and young adults based on their physical conditions enables silver robots to provide customized services for the elderly to counter aging society problems, laying the groundwork for improving elderly welfare systems and automating elderly care systems. Accordingly, skeleton sequences modeling the changes of the human body are converted into pose evolution images (PEIs), and a convolutional neural network (CNN) is trained to classify the elderly and young adults for a single behavior. Then, a heatmap, which is a contributed portion of the inputs, is obtained using a gradient-weighted class activation map (Grad-CAM) for the classified results, and a skeleton-heatmap is obtained through a series of processes for the ease of analysis. Finally, the behavioral characteristics are derived through the difference matching analysis between the domains based on the skeleton-heatmap and RGB video matching analysis. In this study, we present the analysis of the behavioral characteristics of the elderly and young adults based on cognitive science using deep learning and discuss the examples of the analysis. Therefore, we have used the ETRI-Activity3D dataset, which is the largest of its kind among the datasets that have classified the behaviors of young adults and the elderly.

Behavior recognition has applications in automatic crime monitoring, automatic sports video analysis, and context awareness of so-called silver robots. In this study, we employ deep learning to recognize behavior based on body and hand–object interaction regions of interest (ROIs). We propose an ROI-based four-stream ensemble convolutional neural network (CNN). Behavior recognition data are mainly composed of images and skeletons. The first stream uses a pre-trained 2D-CNN by converting the 3D skeleton sequence into pose evolution images (PEIs). The second stream inputs the RGB video into the 3D-CNN to extract temporal and spatial features. The most important information in behavior recognition is identification of the person performing the action. Therefore, if the neural network is trained by removing ambient noise and placing the ROI on the person, feature analysis can be performed by focusing on the behavior itself rather than learning the entire region. Therefore, the third stream inputs the RGB video limited to the body-ROI into the 3D-CNN. The fourth stream inputs the RGB video limited to ROIs of hand–object interactions into the 3D-CNN. Finally, because better performance is expected by combining the information of the models trained with attention to these ROIs, better recognition will be possible through late fusion of the four stream scores. The Electronics and Telecommunications Research Institute (ETRI)-Activity3D dataset was used for the experiments. This dataset contains color images, images of skeletons, and depth images of 55 daily behaviors of 50 elderly and 50 young individuals. The experimental results showed that the proposed model improved recognition by at least 4.27% and up to 20.97% compared to other behavior recognition methods.

This paper is concerned with posture recognition using ensemble convolutional neural networks (CNNs) in home environments. With the increasing number of elderly people living alone at home, posture recognition is very important for helping elderly people cope with sudden danger. Traditionally, to recognize posture, it was necessary to obtain the coordinates of the body points, depth, frame information of video, and so on. In conventional machine learning, there is a limitation in recognizing posture directly using only an image. However, with advancements in the latest deep learning, it is possible to achieve good performance in posture recognition using only an image. Thus, we performed experiments based on VGGNet, ResNet, DenseNet, InceptionResNet, and Xception as pre-trained CNNs using five types of preprocessing. On the basis of these deep learning methods, we finally present the ensemble deep model combined by majority and average methods. The experiments were performed by a posture database constructed at the Electronics and Telecommunications Research Institute (ETRI), Korea. This database consists of 51,000 images with 10 postures from 51 home environments. The experimental results reveal that the ensemble system by InceptionResNetV2s with five types of preprocessing shows good performance in comparison to other combination methods and the pre-trained CNN itself.

We evaluated electrocardiogram (ECG) biometrics using pre-configured models of convolutional neural networks (CNNs) with various time-frequency representations. Biometrics technology records a person’s physical or behavioral characteristics in a digital signal via a sensor and analyzes it to identify the person. An ECG signal is obtained by detecting and amplifying a minute electrical signal flowing on the skin using a noninvasive electrode when the heart muscle depolarizes at each heartbeat. In biometrics, the ECG is especially advantageous in security applications because the heart is located within the body and moves while the subject is alive. However, a few body states generate noisy biometrics. The analysis of signals in the frequency domain has a robust effect on the noise. As the ECG is noise-sensitive, various studies have applied time-frequency transformations that are robust to noise, with CNNs achieving a good performance in image classification. Studies have applied time-frequency representations of the 1D ECG signals to 2D CNNs using transforms like MFCC (mel frequency cepstrum coefficient), spectrogram, log spectrogram, mel spectrogram, and scalogram. CNNs have various pre-configured models such as VGGNet, GoogLeNet, ResNet, and DenseNet. Combinations of the time-frequency representations and pre-configured CNN models have not been investigated. In this study, we employed the PTB (Physikalisch-Technische Bundesanstalt)-ECG and CU (Chosun University)-ECG databases. The MFCC accuracies were 0.45%, 2.60%, 3.90%, and 0.25% higher than the spectrogram, log spectrogram, mel spectrogram, and scalogram accuracies, respectively. The Xception accuracies were 3.91%, 0.84%, and 1.14% higher than the VGGNet-19, ResNet-101, and DenseNet-201 accuracies, respectively.

This paper conducts a comparative analysis of deep models in biometrics using scalogram of electrocardiogram (ECG). A scalogram is the absolute value of the continuous wavelet transform coefficients of a signal. Since biometrics using ECG signals are sensitive to noise, studies have been conducted by transforming signals into a frequency domain that is efficient for analyzing noisy signals. By transforming the signal from the time domain to the frequency domain using the wavelet, the 1-D signal becomes a 2-D matrix, and it could be analyzed at multiresolution. However, this process makes signal analysis morphologically complex. This means that existing simple classifiers could perform poorly. We investigate the possibility of using the scalogram of ECG as input to deep convolutional neural networks of deep learning, which exhibit optimal performance for the classification of morphological imagery. When training data is small or hardware is insufficient for training, transfer learning can be used with pretrained deep models to reduce learning time, and classify it well enough. In this paper, AlexNet, GoogLeNet, and ResNet are considered as deep models of convolutional neural network. The experiments are performed on two databases for performance evaluation. Physikalisch-Technische Bundesanstalt (PTB)-ECG is a well-known database, while Chosun University (CU)-ECG is directly built for this study using the developed ECG sensor. The ResNet was 0.73%—0.27% higher than AlexNet or GoogLeNet on PTB-ECG—and the ResNet was 0.94%—0.12% higher than AlexNet or GoogLeNet on CU-ECG.

We herein propose an EigenECG Network (EECGNet) based on the principal component analysis network (PCANet) for the personal identification of electrocardiogram (ECG) from human biosignal data. The EECGNet consists of three stages. In the first stage, ECG signals are preprocessed by normalization and spike removal. The R peak points in the preprocessed ECG signals are detected. Subsequently, ECG signals are transformed into two-dimensional images to use as the input to the EECGNet. Further, we perform patch-mean removal and PCA algorithm similar to the PCANet from the transformed two-dimensional images. The second stage is almost the same as the first stage, where the mean removal and PCA process are repeatedly performed in the cascaded network. In the final stage, the binary quantization, block sliding, and histogram computation are performed. Thus, this EECGNet performs well without the use of back-propagation to obtain features from the visual content. We constructed a Chosun University (CU)-ECG database from an ECG sensor implemented by ourselves. Further, we used the well-known MIT Beth Israel Hospital (BIH) ECG database. The experimental results clearly reveal the good performance and effectiveness of the proposed method compared with conventional algorithms such as PCA, auto-encoder (AE), extreme learning machine (ELM), and ensemble extreme learning machine (EELM).

In this study, we present a third-order tensor-based multilinear eigenECG (MEECG) and multilinear Fisher ECG (MFECG) for individual identification based on the information obtained by an electrocardiogram (ECG) sensor. MEECG and MFECG are based on multilinear principal component analysis (MPCA) and multilinear linear discriminant analysis (MLDA) in the field of multilinear subspace learning (MSL), respectively. MSL directly extracts features without the vectorization of input data, while MSL extracts features without vectorizing the input data while maintaining most of the correlations shown in the original structure. In contrast with unsupervised linear subspace learning (LSL) techniques such as PCA (Principal Component Analysis) and LDA (Linear Discriminant Analysis), it is less susceptible to small-data problems because it learns more compact and potentially useful representations, and it can efficiently handle large tensors. Here, the third-order tensor is formed by reordering the one-dimensional ECG signal into a two-dimensional matrix, considering the time frame. The MSL consists of four steps. The first step is preprocessing, in which input samples are centered. The second step is initialization, in which eigen decomposition is performed and the most significant eigenvectors are selected. The third step is local optimization, in which input data is applied by eigenvectors from the second step, and new eigenvectors are calculated using the applied input data. The final step is projection, in which the resultant feature tensors after projection are obtained. The experiments are performed on two databases for performance evaluation. The Physikalisch-Technische Bundesanstalt (PTB)-ECG is a well-known database, and Chosun University (CU)-ECG is directly built for this study using the developed ECG sensor. The experimental results revealed that the tensor-based MEECG and MFECG showed good identification performance in comparison to PCA and LDA of LSL.

This paper discusses the classification of horse gaits for self-coaching using an ensemble stacked auto-encoder (ESAE) based on wavelet packets from the motion data of the horse rider. For this purpose, we built an ESAE and used probability values at the end of the softmax classifier. First, we initialized variables such as hidden nodes, weight, and max epoch using the options of the auto-encoder (AE). Second, the ESAE model is trained by feedforward, back propagation, and gradient calculation. Next, the parameters are updated by a gradient descent mechanism as new parameters. Finally, once the error value is satisfied, the algorithm terminates. The experiments were performed to classify horse gaits for self-coaching. We constructed the motion data of a horse rider. For the experiment, an expert horse rider of the national team wore a suit containing 16 inertial sensors based on a wireless network. To improve and quantify the performance of the classification, we used three methods (wavelet packet, statistical value, and ensemble model), as well as cross entropy with mean squared error. The experimental results revealed that the proposed method showed good performance when compared with conventional algorithms such as the support vector machine (SVM).

In this paper, we develop a genetically oriented rule-based Incremental Granular Model (IGM). The IGM is designed using a combination of a simple Linear Regression (LR) model and a local Linguistic Model (LM) to predict the modeling error obtained by the LR. The IGM has been successfully applied to various examples. However, the disadvantage of IGM is that the number of clusters in each context is determined, with the same number, by trial and error. Moreover, a weighting exponent is set to the typical value. In order to solve these problems, the goal of this paper is to design an optimized rule-based IGM with the use of a Genetic Algorithm (GA) to simultaneously optimize the number of cluster centers in each context, the number of contexts, and the weighting exponent. The experimental results regarding a coagulant dosing process in a water purification plant, an automobile mpg (miles per gallon) prediction, and a Boston housing data set revealed that the proposed GA-based IGM showed good performance, when compared with the Radial Basis Function Neural Network (RBFNN), LM, Takagi–Sugeno–Kang (TSK)-Linguistic Fuzzy Model (LFM), GA-based LM, and IGM itself.

In this study, we classify four horse gaits (walk, sitting trot, rising trot, canter) of three breeds of horse (Jeju, Warmblood, and Thoroughbred) using a neuro-fuzzy classifier (NFC) of the Takagi-Sugeno-Kang (TSK) type from data information transformed by a wavelet packet (WP). The design of the NFC is accomplished by using a fuzzy c-means (FCM) clustering algorithm that can solve the problem of dimensionality increase due to the flexible scatter partitioning. For this purpose, we use the rider’s hip motion from the sensor information collected by inertial sensors as feature data for the classification of a horse’s gaits. Furthermore, we develop a coaching system under both real horse riding and simulator environments and propose a method for analyzing the rider’s motion. Using the results of the analysis, the rider can be coached in the correct motion corresponding to the classified gait. To construct a motion database, the data collected from 16 inertial sensors attached to a motion capture suit worn by one of the country’s top-level horse riding experts were used. Experiments using the original motion data and the transformed motion data were conducted to evaluate the classification performance using various classifiers. The experimental results revealed that the presented FCM-NFC showed a better accuracy performance (97.5%) than a neural network classifier (NNC), naive Bayesian classifier (NBC), and radial basis function network classifier (RBFNC) for the transformed motion data.

In this study, an intelligent predictor is designed for predicting the direction of dinghy booms and coaching dinghy sailing using the designed predictor with information obtained from multiple sensors attached to the dinghy and the sailor. For this purpose, we designed a Takagi-Sugeno-Kang-based linguistic model with interval prediction based on fuzzy granulation, which can be realized by using a context-based fuzzy c-means clustering. The sensors used in this study are cameras, GPS, and an anemometer. The GPS and cameras were attached to the dinghy, and the anemometer was installed on a separate boat near the dinghy. Features are extracted from obtained data to interpret them discretely without sequential data. The boom direction was predicted by collecting information from the dinghy driven by an expert wearing a marked suit to predict the optimal direction. The constructed database was randomly divided into a training set (60%) and a validation set (40%) for a 10-fold cross-validation. The experimental results revealed that in the prediction of the dinghy boom direction, the proposed predictor showed performance improvements of 25.2% and 17.9% on the training and validation sets, respectively, when compared to the previous predictors. [ABSTRACT FROM AUTHOR]