Search

One important goal of cognitive science is to understand the mind in terms of its representational and computational capacities, where computational modeling plays an essential role in providing theoretical explanations and predictions of human behavior and mental phenomena. In my research, I have been using computational modeling, together with behavioral experiments and cognitive neuroscience methods, to investigate the information processing mechanisms underlying learning and visual cognition in terms of perceptual representation and attention strategy. In perceptual representation, I have used neural network models to understand how the split architecture in the human visual system influences visual cognition, and to examine perceptual representation development as the results of expertise. In attention strategy, I have developed the Eye Movement analysis with Hidden Markov Models method for quantifying eye movement pattern and consistency using both spatial and temporal information, which has led to novel findings across disciplines not discoverable using traditional methods. By integrating it with deep neural networks (DNN), I have developed DNN+HMM to account for eye movement strategy learning in human visual cognition. The understanding of the human mind through computational modeling also facilitates research on artificial intelligence's (AI) comparability with human cognition, which can in turn help explainable AI systems infer humans' belief on AI's operations and provide human‐centered explanations to enhance human−AI interaction and mutual understanding. Together, these demonstrate the essential role of computational modeling methods in providing theoretical accounts of the human mind as well as its interaction with its environment and AI systems. In this paper, I summarize my research in visual cognition, language processing, and explainable AI to demonstrate the essential role of computational modeling methods in providing theoretical accounts of the human mind as well as its interaction with its environment and AI systems. [ABSTRACT FROM AUTHOR]

The use of multiple raters to label datasets is an established practice in affective computing. The principal goal is to reduce unwanted subjective bias in the labelling process. Unfortunately, this leads to the key problem of identifying a ground truth for training the affect recognition system. This problem becomes more relevant in a sparsely-crossed annotation where each rater only labels a portion of the full dataset to ensure a manageable workload per rater. In this article, we introduce a Multi-Rater Consensus Learning (MRCL) method which learns a representative affect recognition model that accounts for each rater's agreement with the other raters. MRCL combines a multitask learning (MTL) regularizer and a consensus loss. Unlike standard MTL, this approach allows the model to learn to predict each rater's label while explicitly accounting for the consensus among raters. We evaluated our approach on two different datasets based on spontaneous affective body movement expressions for pain behaviour detection and laughter type recognition respectively. The two naturalistic datasets were chosen for the different forms of labelling (different in affect, observation stimuli, and raters) that they together offer for evaluating our approach. Empirical results demonstrate that MRCL is effective for modelling affect from datasets with sparsely-crossed multi-rater annotation. [ABSTRACT FROM AUTHOR]

The interest in automatic emotion recognition and the larger field of Affective Computing has recently gained momentum. The current emergence of large, video-based affect datasets offering rich multi-modal inputs facilitates the development of deep learning-based models for automatic affect analysis that currently holds the state of the art. However, recent approaches to process these modalities cannot fully exploit them due to the use of oversimplified fusion schemes. Furthermore, the efficient use of temporal information inherent to these huge data are also largely unexplored hindering their potential progress. In this work, we propose a multi-modal, sequence-based neural network with gating mechanisms for Valence and Arousal based affect recognition. Our model consists of three major networks: Firstly, a latent-feature generator that extracts compact representations from both modalities that have been artificially degraded to add robustness. Secondly, a multi-task discriminator that estimates both input identity and a first step emotion quadrant estimation. Thirdly, a sequence-based predictor with attention and gating mechanisms that effectively merges both modalities and uses this information through sequence modelling. In our experiments on the SEMAINE and SEWA affect datasets, we observe the impact of both proposed methods with progressive increase in accuracy. We further show in our ablation studies how the internal attention weight and gating coefficient impact our models’ estimates quality. Finally, we demonstrate state of the art accuracy through comparisons with current alternatives on both datasets. [ABSTRACT FROM AUTHOR]

This article investigates classification of emotions from full-body movements by using a novel Convolutional Neural Network-based architecture. The model is composed of two shallow networks processing in parallel when the 8-bit RGB images obtained from time intervals of 3D-positional data are the inputs. One network performs a coarse-grained modelling in the time domain while the other one applies a fine-grained modelling. We show that combining different temporal scales into a single architecture improves the classification results of a dataset composed of short excerpts of the performances of professional dancers who interpreted four affective states: anger, happiness, sadness, and insecurity. Additionally, we investigate the effect of data chunk duration, overlapping, the size of the input images and the contribution of several data augmentation strategies for our proposed method. Better recognition results were obtained when the duration of a data chunk was longer, and this was further improved by applying balanced data augmentation. Moreover, we test our method on other existing motion capture datasets and compare the results with prior art. In all experiments, our results surpassed the state-of-the-art approaches, showing that this method generalizes across diverse settings and contexts. [ABSTRACT FROM AUTHOR]

Emotional voice conversion (EVC) seeks to convert the emotional state of an utterance while preserving the linguistic content and speaker identity. In EVC, emotions are usually treated as discrete categories overlooking the fact that speech also conveys emotions with various intensity levels that the listener can perceive. In this paper, we aim to explicitly characterize and control the intensity of emotion. We propose to disentangle the speaker style from linguistic content and encode the speaker style into a style embedding in a continuous space that forms the prototype of emotion embedding. We further learn the actual emotion encoder from an emotion-labelled database and study the use of relative attributes to represent fine-grained emotion intensity. To ensure emotional intelligibility, we incorporate emotion classification loss and emotion embedding similarity loss into the training of the EVC network. As desired, the proposed network controls the fine-grained emotion intensity in the output speech. Through both objective and subjective evaluations, we validate the effectiveness of the proposed network for emotional expressiveness and emotion intensity control. [ABSTRACT FROM AUTHOR]

Fetal electrocardiogram (FECG) signals contain important information about the conditions of the fetus during pregnancy. Currently, pure FECG signals can only be obtained through an invasive acquisition process, which is life threatening to both mother and fetus. In this study, single-channel ECG signals from the mother’s abdomen are analyzed with the aim of extracting the clean FECG waveform. This is a challenging task due to the very low amplitude of the FECG, various noises involved in the signal acquisition, and the overlap of R waves. To address this problem, we propose a novel convolutional autoencoder (AE) network architecture to learn and extract the FECG patterns. The proposed model is equipped with a dual attention mechanism, composed of squeeze-and-excitation and channel-wise (CW) modules, in the encoder and decoder blocks, respectively. It also benefits from a bidirectional long short-term memory (LSTM) layer. This unique combination allows the proposed network to accurately attend to and extract the FECG signals from abdominal data. Three well-established datasets are considered in our experiments. The obtained results of FECG extraction are promising and confirm the effectiveness of using attention modules within the deep learning model. The results also suggest that the proposed AE network can accurately extract the FECG signals where no information about maternal ECG (MECG) is available. [ABSTRACT FROM AUTHOR]

When viewing omnidirectional images (ODIs), viewers can access different viewports via head movement (HM), which sequentially forms head trajectories in spatial-temporal domain. Thus, head trajectories play a key role in modeling human attention on ODIs. In this paper, we establish a large-scale dataset collecting 21,600 head trajectories on 1,080 ODIs. By mining our dataset, we find two important factors influencing head trajectories, i.e., temporal dependency and subject-specific variance. Accordingly, we propose a novel approach integrating hierarchical Bayesian inference into long short-term memory (LSTM) network for head trajectory prediction on ODIs, which is called HiBayes-LSTM. In HiBayes-LSTM, we develop a mechanism of Future Intention Estimation (FIE), which captures the temporal correlations from previous, current and estimated future information, for predicting viewport transition. Additionally, a training scheme called Hierarchical Bayesian inference (HBI) is developed for modeling inter-subject uncertainty in HiBayes-LSTM. For HBI, we introduce a joint Gaussian distribution in a hierarchy, to approximate the posterior distribution over network weights. By sampling subject-specific weights from the approximated posterior distribution, our HiBayes-LSTM approach can yield diverse viewport transition among different subjects and obtain multiple head trajectories. Extensive experiments validate that our HiBayes-LSTM approach significantly outperforms 9 state-of-the-art approaches for trajectory prediction on ODIs, and then it is successfully applied to predict saliency on ODIs. [ABSTRACT FROM AUTHOR]

This paper presents a novel fault detection approach for industrial batch processes. The batch processes under consideration are characterized by the interaction between discrete system modes and non-stationary continuous dynamics. Therefore, a stochastic hybrid process model (SHPM) is introduced, where process variables are modeled as time-variant Gaussian distributions, which depend on hidden system modes. Transitions between the system modes are assumed to be either autonomous or to be triggered by observable events such as on/off signals. The model parameters are determined from training data using expectation-maximization techniques. A new fault detection algorithm is proposed, which assesses the likelihoods of sensor signals on the basis of the stochastic hybrid process model. Evaluation of the proposed fault detection system has been conducted for a penicillin production process, with the results showing a significant improvement over the existing baseline methods.Note to Practitioners—Automatic fault detection makes it possible to limit the effects of faults by taking countermeasures at an early stage. In this work, a data-driven fault detection method for industrial batch processes is proposed, in which the underlying process model is learned from training data. The proposed fault detection system can be used for various industrial batch processes without the need for complex and error-prone manual configuration. In contrast to many other data-driven approaches such as neural networks, only a few process cycles are required to create a robust process model. It should be noted that in data-driven fault detection methods, the training data should cover a large part of the process states that occur during error-free process cycles. The developed method is therefore particularly suitable for cyclical processes, which, however, can have alternative process paths and variability between the process cycles. [ABSTRACT FROM AUTHOR]

In medical image analysis (MIA) and computer-assisted surgery (CAS), aligning two multiple point sets (PSs) together is an essential but also a challenging problem. For example, rigidly aligning multiple point sets into one common coordinate frame is a prerequisite for statistical shape modelling (SSM). Accurately aligning the pre-operative space with the intra-operative space in CAS is very crucial to successful interventions. In this article, we formally formulate the multiple generalized point set registration problem (MGPSR) in a probabilistic manner, where both the positional and the normal vectors are used. The six-dimensional vectors consisting of both positional and normal vectors are called as generalized points. In the formulated model, all the generalized PSs to be registered are considered to be the realizations of underlying unknown hybrid mixture models (HMMs). By assuming the independence of the positional and orientational vectors (i.e., the normal vectors), the probability density function (PDF) of an observed generalized point is computed as the product of Gaussian and Fisher distributions. Furthermore, to consider the anisotropic noise in surgical navigation, the positional error is assumed to obey a multi-variate Gaussian distribution. Finally, registering PSs is formulated as a maximum likelihood (ML) problem, and solved under the expectation maximization (EM) technique. By using more enriched information (i.e., the normal vectors), our algorithm is more robust to outliers. By treating all PSs equally, our algorithm does not bias towards any PS. To validate the proposed approach, extensive experiments have been conducted on surface points extracted from CT images of (i) a human femur bone model; (ii) a human pelvis bone model. Results demonstrate our algorithm’s high accuracy, robustness to noise and outliers. Note to Practitioners—This paper was motivated by solving the problem of registering two or more PSs. Most existing registration approaches use only the positional information associated with each point, and thus lacks robustness to noise and outliers. Three significant improvements are brought by our proposed approach. First, the normal vectors that can be extracted from the point sets are utilized in the registration. Second, the positional error distribution is assumed to be anisotropic and inhomogeneous. Third, all the PSs to be registered are treated equally that means no PS is considered as the model one. The registration problem is cast into a maximum likelihood (ML) problem and solved under the expectation maximization (EM) framework. We have demonstrated through extensive experiments that the proposed registration approach achieves significantly improved accuracy, robustness to noise and outliers. The algorithm is particularly suitable for biomedical applications involving the registration procedures, such as image-guided surgery. [ABSTRACT FROM AUTHOR]

In this paper, we propose a novel deep Efficient Relational Sentence Ordering Network (referred to as ERSON) by leveraging pre-trained language model in both encoder and decoder architectures to strengthen the coherence modeling of the entire model. Specifically, we first introduce a divide-and-fuse BERT (referred to as DF-BERT), a new refactor of BERT network, where lower layers in the improved model encode each sentence in the paragraph independently, which are shared by different sentence pairs, and the higher layers learn the cross-attention between sentence pairs jointly. It enables us to capture the semantic concepts and contextual information between the sentences of the paragraph, while significantly reducing the runtime and memory consumption without sacrificing the model performance. Besides, a Relational Pointer Decoder (referred to as RPD) is developed, which utilizes the pre-trained Next Sentence Prediction (NSP) task of BERT to capture the useful relative ordering information between sentences to enhance the order predictions. In addition, a variety of knowledge distillation based losses are added as auxiliary supervision to further improve the ordering performance. The extensive evaluations on Sentence Ordering, Order Discrimination, and Multi-Document Summarization tasks show the superiority of ERSON to the state-of-the-art ordering methods. [ABSTRACT FROM AUTHOR]

This brief presents a low-power attention-based automatic speech recognition (ASR) processor achieving real-time recognition capability. The proposed attention window algorithm, compact end-to-end neural-network topology, and efficient computation dataflow effectively minimize the hardware complexity and power consumption, enabling a fully integrated low-power ASR processor solution without the necessity of any off-chip memory resource. The proposed design techniques reduced 98.9% weight memory and 92.1% power consumption with minimal degradation of 2.24% in recognition accuracy. The proposed ASR processor operates at 100MHz with 1.7mW at 0.9V, demonstrating 2x and 1.68x performance improvements in speed and power, respectively, compared to the previous ASR designs that require additional supports of off-chip memory or external decoder. [ABSTRACT FROM AUTHOR]

Few-shot learning under the $N$ N -way $K$ K -shot setting (i.e., $K$ K annotated samples for each of $N$ N classes) has been widely studied in relation extraction (e.g., FewRel) and image classification (e.g., Mini-ImageNet). Named entity recognition (NER) is typically framed as a sequence labeling problem where the entity classes are inherently entangled together because the entity number and classes in a sentence are not known in advance, leaving the $N$ N -way $K$ K -shot NER problem so far unexplored. In this paper, we first formally define a more suitable $N$ N -way $K$ K -shot setting for NER. Then we propose FewNER, a novel meta-learning approach for few-shot NER. FewNER separates the entire network into a task-independent part and a task-specific part. During training in FewNER, the task-independent part is meta-learned across multiple tasks and the task-specific part is learned for each individual task in a low-dimensional space. At test time, FewNER keeps the task-independent part fixed and adapts to a new task via gradient descent by updating only the task-specific part, resulting in it being less prone to overfitting and more computationally efficient. Compared with pre-trained language models (e.g., BERT and ELMo) which obtain the transferability in an implicit manner (i.e., relying on large-scale corpora), FewNER explicitly optimizes the capability of “learning to adapt quickly” through meta-learning. The results demonstrate that FewNER achieves state-of-the-art performance against nine baseline methods by significant margins on three adaptation experiments (i.e., intra-domain cross-type, cross-domain intra-type and cross-domain cross-type). [ABSTRACT FROM AUTHOR]

The Internet of Things (IoT), coupled with the edge computing paradigm, is enabling several pervasive networked applications with stringent real-time requirements, such as telemedicine and haptic telecommunications. Recent advances in network virtualization and artificial intelligence are helping solve network latency and capacity problems, learning from several states of the network stack. However, despite such advances, a network architecture able to meet the demands of next-generation networked applications with stringent real-time requirements still has untackled challenges. In this paper, we argue that only using network (or transport) layer information to predict traffic evolution and other network states may be insufficient, and a more holistic approach that considers predictions of application-layer states is needed to repair the inefficiencies of the TCP/IP architecture. Based on this intuition, we present the design and implementation of Reparo. At its core, the design of our solution is based on the detection of a packet loss and its restoration using a Hidden Markov Model (HMM) empowered with adversarial autoencoders. In our evaluation, we considered a telemedicine use case, specifically a telepathology session, in which a microscope is controlled remotely in real-time to assess histological imagery. Our results confirm that the use of adversarial autoencoders enhances the accuracy of the prediction method satisfying our telemedicine application’s requirements with a notable improvement in terms of throughput and latency perceived by the user. [ABSTRACT FROM AUTHOR]

Network slicing markets have the potential to increase significantly the utilization of virtualized network resources and facilitate the low-cost deployment of over-the-top services. However, their success is conditioned on the service providers (SPs) being able to bid effectively for the virtualized resources. In this paper, we consider a hybrid advance-reservation and spot slice market and study how the SPs should reserve resources to maximize their services’ performance while not violating a time-average budget threshold. We consider this problem in its general form where the SP demand and slice prices are time-varying and revealed only after the reservations are decided. We develop a learning-based framework, using the theory of online convex optimization, that allows the SP to employ a no-regret reservation policy, i.e., achieve the same performance with an oracle that has full access to all future demand and prices. We extend the framework to the scenario where the SP decides dynamically its slice orchestration and hence needs to learn the performance-maximizing resource composition; and we further develop a mixed-time scale scheme that allows the SP to leverage spot-market information that is revealed between successive reservations. The proposed learning framework is evaluated using representative simulation scenarios that highlight its efficacy as well as the impact of key system and algorithm parameters. [ABSTRACT FROM AUTHOR]

The accurate detection of physiologically-related events in photopletismographic (PPG) and phonocardiographic (PCG) signals, recorded by wearable sensors, is mandatory to perform the estimation of relevant cardiovascular parameters like the heart rate and the blood pressure. However, the measurement performed in uncontrolled conditions without clinical supervision leaves the detection quality particularly susceptible to noise and motion artifacts. This work proposes a new fully-automatic computational framework, based on convolutional networks, to identify and localize fiducial points in time as the foot, maximum slope and peak in PPG signal and the S1 sound in the PCG signal, both acquired by a custom chest sensor, described recently in the literature by our group. The event detection problem was reframed as a single hybrid regression-classification problem entailing a custom neural architecture to process sequentially the PPG and PCG signals. Tests were performed analysing four different acquisition conditions (rest, cycling, rest recovery and walking). Cross-validation results for the three PPG fiducial points showed identification accuracy greater than 93 % and localization error (RMSE) less than 10 ms. As expected, cycling and walking conditions provided worse results than rest and recovery, however reaching an accuracy greater than 90 % and a localization error less than 15 ms. Likewise, the identification and localization error for S1 sound were greater than 90 % and less than 25 ms. Overall, this study showcased the ability of the proposed technique to detect events with high accuracy not only for steady acquisitions but also during subject movements. We also showed that the proposed network outperformed traditional Shannon-energy-envelope method in the detection of S1 sound, reaching detection performance comparable to state of the art algorithms. Therefore, we argue that coupling chest sensors and deep learning processing techniques may disclose wearable devices to unobtrusively acquire health information, being less affected by noise and motion artifacts. [ABSTRACT FROM AUTHOR]

Temporal information is critical for routing computation in the vehicular network. It plays a vital role in the vehicular network. Till now, most existing routing schemes in vehicular networks consider the networks as a sequence of static graphs. We need to find an appropriate method to process temporal information into routing computation. Thus, in this paper, we propose a routing algorithm based on the Hidden Markov Model (HMM) and temporal graph, namely, Prediction-Based Temporal Graph Routing Algorithm (PT-GROUT). This new algorithm considers the vehicular network as a temporal graph, in which each data transmission as an edge has its specific temporal information. To better capture the temporal information, we select Software-Defined Vehicular Network (SDVN) as our network architecture, which is a preferred architecture for processing the temporal graph regarding the vehicular network since all vehicle statuses can be easily managed. To compute the future routing path accurately and efficiently, the future temporal graph is predicted by applying HMM, in which we model the current vehicular network with dynamic programming and greedy strategies. With the temporal information and reasonable setting of HMM, PT-GROUT can better evaluate the vehicular network and discover the evolution of the internal structure of the network. The optimal routing path can be achieved more efficiently. The simulation results demonstrate that PT-GROUT can substantially improve the computation efficiency and reduce packet loss and delivery delay compared with its counterparts. [ABSTRACT FROM AUTHOR]

Anomaly (outlier) detection is one of the most important problems of modern data analysis. The sources of anomalies are varying. They can be the results of database users’ mistakes, operational errors, or just missing values. The problem is very important because of the fast growth of large datasets. Therefore, in this article, we present detailed results of work on the concept of granular computing-based approach to anomaly detection, classification, and gradation. The aim of the study is to introduce an innovative solution that allows the use of information granules to identify and classify anomalies. The novelty of the proposed solution consists in the use of fuzzy semantics implied by the statistical properties of the data considered. Moreover, instead of the classic approach to detecting anomalies in the data, it is proposed to determine the degree of anomaly for the data transformed to the new resulting state space. Thanks to the use of an innovative approach using the universal descriptor space, it is possible to determine the degree of anomaly, and by using various aggregation methods one can also specify its type. [ABSTRACT FROM AUTHOR]

This brief investigates the event-triggered output feedback control problem for nonlinear Markov jump systems (NMJSs) with unknown probabilities. The NMJSs with parameter uncertainties are characterized by using the interval type-2 (IT2) fuzzy model. An adaptive event-triggered strategy is utilized to decide when to transmit, which can reduce communication costs. The output feedback controller is designed, where the modes can be allowed different with the system, and the conditional probability matrix does not require completely known. Then, an event-based asynchronous output feedback control approach is proposed for obtaining the required result. With the aid of an example, we demonstrate the effectiveness of the event-based asynchronous output control algorithms. [ABSTRACT FROM AUTHOR]

Aircraft operations in the terminal area rely heavily on voice communications between pilots and air traffic controllers. This paper proposes a novel aircraft trajectory simulation framework by guiding the trajectory simulation following the voice command from controllers. Bayesian model selection is used for checking pilot compliances to controller commands with observed trajectories. This framework is named as Voice Communication-Augmented Simulation. The goal of the proposed study is to enable accurate trajectory predictions. The framework can act as a computer assistant for controllers to monitor pilot compliances and ensure safe operations. The proposed method is tested and validated with actual trajectory data from Sherlock Data Warehouse. The tests showed that the proposed framework can accurately simulate and monitor the flight level change of aircraft and update the approach procedure. [ABSTRACT FROM AUTHOR]

Hallucinations are vivid and transient experiences of objects, such as images or sounds, that occur in the absence of a corresponding stimulus. 1,2,3,4,5,6,7,8,9 To understand the neurocomputational mechanisms of hallucinations, cognitive neuroscience has focused on experiments that induce false alarms (FAs) in healthy participants, 1,2,3,4,5,9 psychosis-prone individuals, 1,3,4 and patients diagnosed with schizophrenia. 5 FAs occur when participants make decisions about difficult-to-detect stimuli and indicate the presence of a signal that was, in fact, not presented. Since FAs are, at heart, reports, they must meet two criteria to serve as an experimental proxy for hallucinations: first, FAs should reflect perceptual states that are characterized by specific contents 10,11,12 (criterion 1). Second, FAs should occur on a timescale compatible with the temporal dynamics of hallucinations 13,14 (criterion 2). In this work, we combined a classification image approach 15 with hidden Markov models 16 to show that FAs can match the perceptual and temporal characteristics of hallucinations. We asked healthy human participants to discriminate visual stimuli from noise and found that FAs were more likely to occur during an internal mode of sensory processing, a minute-long state of the brain during which perception is strongly biased toward previous experiences 17 (serial dependency). Our results suggest that hallucinations are driven by dynamic predictive templates that transform noise into transient, coherent, and meaningful perceptual experiences. • Perception alternates between external and internal modes • In the external mode, perception follows the sensory signals • In the internal mode, perception is determined by serial dependencies • False alarms are more likely to occur during the internal mode Weilnhammer et al. show that false alarms are more likely to occur during the internal mode, a minute-long state of the brain during which perception is strongly biased toward previous experiences. [ABSTRACT FROM AUTHOR]

The magnetic field calculation for axial flux motors (AFMs) is always intricate and time-consuming. To this end, a fast quasi-three-dimensional (3D) analytical model is proposed in this paper for magnetic field prediction in AFMs. The 3D magnetic field problem can be simplified into a two-dimensional (2D) one by dividing the AFM into several annular slices in the radial direction. Then, for the first time, the harmonic modeling method (HMM) in Cartesian coordinates is adopted to solve the magnetic field in these slices cut from the AFM. The corresponding electromagnetic parameters of the AFM can be acquired subsequently. The critical step for the field prediction of using HMM in Cartesian coordinates is to adopt a variable substitution method when solving the partial differential equations. Next, the magnetic saturation of soft-magnetic material within AFMs is considered by adopting an iterative approach. Ultimately, the effectiveness of the proposed quasi-3D analytical model is validated by the nonlinear finite element analysis (FEA). The proposed analytical model is computationally efficient, which makes it suitable for the preliminary design of AFMs. [ABSTRACT FROM AUTHOR]

Extracting the specific category of the players, such as the malignant Bot, from the huge log data of the massive multiplayer online role playing games, denoted as MMORPGs, is an important basic task in game security and personal recommendation. In this article, we propose a parallel semisupervised framework to categorize specific game players with a few label-known target samples, which are denoted as bait players. Our approach first presents a feature representation model based on the players’ level granularity, which can acquire aligned feature representations in the lower dimensional space from the players’ original action sequences. Then, we propose a semisupervised clustering method, extended from the bisecting $k$ -means model, to extract the specified players with the help of those bait players. Due to massive amounts of game log data, the computation complexity is an extreme challenge to implement our feature representation and semisupervised extraction approaches. We also propose a hierarchical parallelism framework, which allows the data to be computed horizontally and vertically simultaneously and enables varied parallel combinations for the steps of our semisupervised categorization approach. The comparable experiments on real-world MMORPGs’ log data, containing more than 465 Gbytes and million players, are carried out to demonstrate the effectiveness and efficiency of our proposed approach compared with the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Predicting individual vehicles’ future locations is of great significance in location-based services. Regularity and preference are two predominant features of individual vehicles for location prediction. However, these two features cannot be adequately captured with existing models based on raw trajectories, due to their oversimplified view. Modeling regularity and preference entails the following challenges: 1) how to design features to accurately reflect regularity and preference with raw trajectories; and 2) how to capture regularity and preference effectively based on sparse travel behaviors. To that end, first, we design four features to represent travel regularity and preference by conducting an in-depth correlation analysis on individual vehicle trajectories. Then, we incorporate these four features in context and propose a deep neural network to jointly capture regularity and preference via explicitly retrieving the context. Specifically, our proposed model extends LSTM with memory to store all the output hidden states, which provides long-term information for retrieval to overcome the vanishing sequential dependency in sparse scenarios. To fully capture regularity and preference for prediction, a backtracking attention mechanism is designed to aggregate all the relevant historical hidden states in memory with different weights based on the regularity and preference similarity. The weighted aggregation produces a new hidden state, which is used for the final prediction. Experiments on three real-world vehicle trajectory datasets containing over 10,000 individual vehicles show that our proposed model outperforms the state-of-the-art models by 7%–10% in terms of prediction accuracy. [ABSTRACT FROM AUTHOR]

Over their first years of life, children learn not just the words of their native languages, but how to use them to communicate. Because manual annotation of communicative intent does not scale to large corpora, our understanding of communicative act development is limited to case studies of a few children at a few time points. We present an approach to automatic identification of communicative acts using a hidden topic Markov model, applying it to the conversations of English‐learning children in the CHILDES database. We first describe qualitative changes in parent–child communication over development, and then use our method to demonstrate two large‐scale features of communicative development: (a) children develop a parent‐like repertoire of our model's communicative acts rapidly, their learning rate peaking around 14 months of age, and (b) this period of steep repertoire change coincides with the highest predictability between parents' acts and children's, suggesting that structured interactions play a role in learning to communicate. Characterizing the dynamics of children's conversations across development, using a Hidden Topic Markov Model to classify parents' and children's communicative acts and analyze their sequential structure in conversation. [ABSTRACT FROM AUTHOR]

Spatial structured models are predictive models that capture dependency structure between samples based on their locations in the space. Learning such models plays an important role in many geoscience applications such as water surface mapping, but it also poses significant challenges due to implicit dependency structure in continuous space and high computational costs. Existing models often assume that the dependency structure is based on either spatial proximity or network topology, and thus cannot incorporate complex dependency structure such as contour and flow direction on a 3D potential surface. To fill the gap, we recently proposed a novel spatial structured model called hidden Markov contour tree (HMCT), which generalizes the traditional hidden Markov model from a total order sequence to a partial order polytree. HMCT also advances existing work on hidden Markov trees through capturing complex contour structures on a 3D surface. We proposed efficient model construction and learning algorithms. This paper extends our initial HMCT model into a post-processor that can refine the classified results from other existing models. We analyzed the theoretical properties of the extended model. Evaluations on real-world flood mapping datasets show that HMCT outperforms multiple baseline methods in classification performance and the HMCT can also effectively enhance the results of other baseline methods. Computational experiments also show that HMCT is scalable to large data sizes (e.g., classifying millions of samples in seconds). [ABSTRACT FROM AUTHOR]

The problem of quickest change detection (QCD) in anonymous heterogeneous sensor networks is studied. There are $n$ heterogeneous sensors and a fusion center. The sensors are clustered into $K$ groups, and different groups follow different data-generating distributions. At some unknown time, an event occurs in the network and changes the data-generating distribution of the sensors. The goal is to detect the change as quickly as possible, subject to false alarm constraints. The anonymous setting is studied, where at each time step, the fusion center receives $n$ unordered samples, and the fusion center does not know which sensor each sample comes from, and thus does not know its exact distribution. A simple optimality proof is first derived for the mixture likelihood ratio test, which was constructed and proved to be optimal for the non-sequential anonymous setting in (Chen et al., 2019). For the QCD problem, a mixture CuSum algorithm is further constructed, and is further shown to be optimal under Lorden’s criterion. For large networks, a computationally efficient test is proposed and a novel theoretical characterization of its false alarm rate is developed. Numerical results are provided to validate the theoretical results. [ABSTRACT FROM AUTHOR]

Detecting unusual behaviors of insiders on enterprise resource planning (ERP) systems is one of the essential parts to reduce the risks of threatening and abusing enterprise resources by insiders. Many approaches to detect the behaviors based on rule-based systems and stochastic processes are currently limited to empirical monitoring using manually established algorithms or probabilistic boundaries. Those approaches need prior knowledge such as user permission guideline and process data characteristics. Unfortunately, obtaining prior knowledge is hard in practice, and these are not appropriate to detect atypical unusual behavior which can not be clearly defined using heuristic rules. Therefore, in this article, we propose a novel framework for unusual insider behavior detection (UIBD) for ERP systems. The proposed framework initially derives a discriminative model for normal behavior samples, and UIBD is conducted by computing an error using the model. Since the model is compiled using normal samples only, the error of unusual samples would be larger than normal ones. To derive a robust normal behavior model, we present adversarial recurrent autoencoder (ARAE). To demonstrate the efficiency of the proposed framework based on ARAE, we conduct experiments using a dataset composed of insider behaviors defined by sequences of security audit logs of ERP systems operating in real-world enterprises. The experimental results show that the proposed framework with ARAE can successfully detect unusual insider behaviors and outperform other methods to detect unusual insider behavior or threatening. [ABSTRACT FROM AUTHOR]

Objective: This research focuses on studying the clinical decision-making strategies of expert and novice prosthetists for different case complexities. Background: With an increasing global amputee population, there is an urgent need for improved amputee care. However, current prosthetic prescription standards are based on subjective expertise, making the process challenging for novices, specifically during complex patient cases. Hence, there is a need for studying the decision-making strategies of prosthetists. Method: An interactive web-based survey was developed with two case studies of varying complexities. Navigation between survey pages and time spent were recorded for 28 participants including experts (n = 20) and novices (n = 8). Using these data, decision-making strategies, or patterns of decisions, during prosthetic prescription were derived using hidden Markov modeling. A qualitative analysis of participants' rationale regarding decisions was used to add a deep contextualized understanding of decision-making strategies derived from the quantitative analysis. Results: Unique decision-making strategies were observed across expert and novice participants. Experts tended to focus on the personal details, activity level, and state of the residual limb prior to prescription, and this strategy was independent of case complexity. Novices tended to change strategies dependent upon case complexity, fixating on certain factors when case complexity was high. Conclusion: The decision-making strategies of experts stayed the same across the two cases, whereas the novices exhibited mixed strategies. Application: By modeling the decision-making strategies of experts and novices, this study builds a foundation for development of an automated decision-support tool for prosthetic prescription, advancing novice training, and amputee care. [ABSTRACT FROM AUTHOR]

Lithology interpretation is important for understanding subsurface properties. Yet, the common manual well log interpretation is usually with low efficiency and bad consistency. Therefore, the automatic well log interpretation tools based on machine learning and deep learning have been developed. Although the state-of-the-art sophisticated models can show fine interpretation performance with acceptable accuracies, “blind” tests do not always exhibit satisfactory results because of the complexity of lithology interpretation with respect to subsurface rock properties and the data-labeling quality. To solve this generalization challenge, we propose to leverage the parameterized quantum circuits in the deep-learning model. The quantum computing takes advantages of the superposition and entanglement quantum systems, which could potentially endow the generalization power or capability to the deep-learning model. Using the proposed quantum-enhanced deep-learning (QEDL) model, we have tested the model performance on field well log data from different wells. Compared with the classic fine convolutional neural network (CNN) model and the long short-term memory (LSTM) model, the proposed QEDL model achieves comparable model performance with a clearly improved generalization power for interpreting both thin and thick lithology layers. In addition, because of the quantum circuit structure, the QEDL model needs much fewer model parameters than LSTM and CNN models, i.e., the QEDL parameter number in our study can be approximately 75% less than that of LSTM and 89% less than that of CNN. [ABSTRACT FROM AUTHOR]

The design of methods for inference from time sequences has traditionally relied on statistical models that describe the relation between a latent desired sequence and the observed one. A broad family of model-based algorithms have been derived to carry out inference at controllable complexity using recursive computations over the factor graph representing the underlying distribution. An alternative model-agnostic approach utilizes machine learning (ML) methods. Here we propose a framework that combines model-based algorithms and data-driven ML tools for stationary time sequences. In the proposed approach, neural networks are developed to separately learn specific components of a factor graph describing the distribution of the time sequence, rather than the complete inference task. By exploiting stationary properties of this distribution, the resulting approach can be applied to sequences of varying temporal duration. Learned factor graphs can be realized using compact neural networks that are trainable using small training sets, or alternatively, be used to improve upon existing deep inference systems. We present an inference algorithm based on learned stationary factor graphs, which learns to implement the sum-product scheme from labeled data, and can be applied to sequences of different lengths. Our experimental results demonstrate the ability of the proposed learned factor graphs to learn from small training sets to carry out accurate inference for sleep stage detection using the Sleep-EDF dataset, as well as for symbol detection in digital communications with unknown channels. [ABSTRACT FROM AUTHOR]

Performance-score synchronization is an integral task in signal processing, which entails generating an accurate mapping between an audio recording of a performance and the corresponding musical score. Traditional synchronization methods compute alignment using knowledge-driven and stochastic approaches, and are typically unable to generalize well to different domains and modalities. We present a novel data-driven method for structure-aware performance-score synchronization. We propose a convolutional-attentional architecture trained with a custom loss based on time-series divergence. We conduct experiments for the audio-to-MIDI and audio-to-image alignment tasks pertained to different score modalities. We validate the effectiveness of our method via ablation studies and comparisons with state-of-the-art alignment approaches. We demonstrate that our approach outperforms previous synchronization methods for a variety of test settings across score modalities and acoustic conditions. Our method is also robust to structural differences between the performance and score sequences, which is a common limitation of standard alignment approaches. [ABSTRACT FROM AUTHOR]

Network Function Virtualization (NFV) has been emerging as an appealing solution that transforms from dedicated hardware implementations to software instances running in a virtualized environment. In NFV, the requested service is implemented by a sequence of Virtual Network Functions (VNF) that can run on generic servers by leveraging the virtualization technology. These VNFs are pitched with a predefined order, and it is also known as the Service Function Chaining (SFC). Considering that the delay and resiliency are two important Service Level Agreements (SLA) in a NFV service, in this paper, we first investigate how to quantitatively model the traversing delay of a flow in both totally ordered and partially ordered SFCs. Subsequently, we study how to calculate the VNF placement availability mathematically for both unprotected and protected SFCs. After that, we study the delay-sensitive Virtual Network Function placement and routing problem with and without resiliency concerns. We prove that this problem is NP-hard under two cases. We subsequently propose an exact Integer Nonlinear Programming (INLP) formulation and an efficient heuristic for this problem in each case. Finally, we evaluate the proposed algorithms in terms of acceptance ratio, average number of used nodes and total running time via extensive simulations. [ABSTRACT FROM AUTHOR]

We propose a multi-task ensemble framework that jointly learns multiple related problems. The ensemble model aims to leverage the learned representations of three deep learning models (i.e., CNN, LSTM and GRU) and a hand-crafted feature representation for the predictions. Through multi-task framework, we address four problems of emotion and sentiment analysis, i.e., “emotion classification & intensity”, “valence, arousal & dominance for emotion”, “valence & arousal for sentiment”, and “3-class categorical & 5-class ordinal classification for sentiment”. The underlying problems cover two granularity (i.e., coarse-grained and fine-grained) and a diverse range of domains (i.e., tweets, Facebook posts, news headlines, blogs, letters etc.). Experimental results suggest that the proposed multi-task framework outperforms the single-task frameworks in all experiments. [ABSTRACT FROM AUTHOR]

Semisupervised band selection (BS) technique plays an important role in processing hyperspectral images (HSIs) because of its superiority of using the limited labeled data and plentiful unlabeled data to select the discriminative and informative feature, which copes with the problem of high-dimensional and scare labeled samples of HSIs. Among semisupervised BS models, graph-based methods are superior to others in many situations and have received more and more attention. However, traditional graph-based models construct a similarity matrix and select valuable bands independently. The similarity matrix remains constant, which will damage the local manifold structure and lead to a suboptimal result. To solve this problem, we propose a semisupervised band selection with an optimal graph (BSOG) approach, which performs BS and local structure learning simultaneously. Instead of fixing the input similarity matrix, the similarity matrix is updated constantly to learn a better local structure. Besides, the learned similarity matrix is adaptive. Then, the optimal band subset can be selected by analyzing the obtained projection matrix $\boldsymbol W$. An efficient and simple optimization algorithm is proposed to solve this model. Experiments on four real HSIs data validate the effectiveness of the proposed model. [ABSTRACT FROM AUTHOR]

Radar radio source (RRS) recognition plays an important role in the fields of military electronic support systems (ESM) and civilian autonomous driving. The rapid development of machine learning technology, especially deep learning, has effectively and efficiently improved RRS intelligent recognition performances when operating in the increasingly complex electromagnetic environment. However, the data sampling limitation and computation cost are still severe challenges in real RRS recognition scenarios. In this paper, we propose a novel network based on meta-transfer learning, called RRSARNet, to achieve effective adaptive RRS recognition in the context of low signal-to-noise ratio (SNR). First, by using the short-time Fourier transform, a six-type small samples RRS simulation dataset with different SNR levels is constructed. Then, a novel RRSARNet, based on metric learning, is proposed, which consists of a four-layer embedding module and a four-layer relational module. Finally, the RRS dataset is divided into training, supporting and testing subsets, which are used to train and test the RRSARNet in a meta-transfer learning method. Experiments on the RRS dataset show that the proposed RRSARNet can achieve an overall accuracy (OA) above 96% and 99% when the SNR is above −15 dB and −10 dB, respectively. Even when the SNR is -30 dB, OA can reach more than 70%. For 5-way 1-shot and 5-way 5-shot experiments, the inference time of an image is about 0.043 and 0.140 milliseconds, respectively. Besides, experiments on the RRS simulation dataset and the two benchmark datasets, the RRSARNet performs better or more competitive than many existing state-of-the-art technologies in terms of recognition accuracy. [ABSTRACT FROM AUTHOR]

In this paper, we propose MiT: a novel multi-view transformer model for 3D/4D facial affect recognition. MiT incorporates patch and position embeddings from various patches of multi-views and uses them for learning various facial muscle movements to showcase an effective recognition performance. We also propose a multi-view loss function that is not only gradient-friendly, and hence speeds up the gradient computation during back-propagation, but it also leverages the correlation associated with the underlying facial patterns among multi-views. Additionally, we offer multi-view weights that are trainable and learnable, and help substantially in training. Finally, we equip our model with distributed performance for faster learning and computational convenience. With the help of extensive experiments, we show that our model outperform the existing methods on widely-used datasets for 3D/4D FER. [ABSTRACT FROM AUTHOR]

The growing use of electronic health records in the medical domain results in generating a large amount of medical data that is stored in the form of clinical notes. These clinical notes are enriched with clinical entities like disease, treatment, tests, drugs, genes, and proteins. The extraction of clinical entities from clinical notes is a challenging task as clinical notes are written in the form of natural language. The extraction of clinical entities has many useful applications such as clinical notes analysis, medical data privacy, decision support systems, and disease analysis. Although various machine learning and deep learning models are developed to extract clinical entities from clinical notes, developing an accurate model is still challenging. This study presents a novel deep learning-based technique to extract the clinical entities from clinical notes. The proposed model uses local and global context to extract clinical entities in contrast to existing models that use only global context. The combination of CNN, Bi-LSTM, and CRF with non-complex embedding (proposed model) outperforms existing models by a margin of $4-10\%$ and $5-12\%$ in terms of F1-score on i2b2-2010 and i2b2-2012 data. The accurate detection of clinical entities can be helpful in the privacy preservation of medical data that increases the user's and medical organization's trust in sharing medical data. [ABSTRACT FROM AUTHOR]

Recurrent neural network (RNN) has become popular for human motion prediction thanks to its ability to capture temporal dependencies. However, it has limited capacity in modeling the complex spatial relationship in the human skeletal structure. In this work, we present a novel diffusion convolutional recurrent predictor for spatial and temporal movement forecasting, with multi-step random walks traversing bidirectionally along an adaptive graph to model interdependency among body joints. In the temporal domain, existing methods rely on a single forward predictor with the produced motion deflecting to the drift route, which leads to error accumulations over time. We propose to supplement the forward predictor with a forward discriminator to alleviate such motion drift in the long term under adversarial training. The solution is further enhanced by a backward predictor and a backward discriminator to effectively reduce the error, such that the system can also look into the past to improve the prediction at early frames. The two-way spatial diffusion convolutions and two-way temporal predictors together form a quadruple network. Furthermore, we train our framework by modeling the velocity from observed motion dynamics instead of static poses to predict future movements that effectively reduces the discontinuity problem at early prediction. Our method outperforms the state of the arts on both 3D and 2D datasets, including the Human3.6M, CMU Motion Capture and Penn Action datasets. The results also show that our method correctly predicts both high-dynamic and low-dynamic moving trends with less motion drift. [ABSTRACT FROM AUTHOR]

Detection, parsing, and future predictions on sequence data (e.g., videos) require the algorithms to capture non-Markovian and compositional properties of high-level semantics. Context-free grammars are natural choices to capture such properties, but traditional grammar parsers (e.g., Earley parser) only take symbolic sentences as inputs. In this paper, we generalize the Earley parser to parse sequence data which is neither segmented nor labeled. Given the output of an arbitrary probabilistic classifier, this generalized Earley parser finds the optimal segmentation and labels in the language defined by the input grammar. Based on the parsing results, it makes top-down future predictions. The proposed method is generic, principled, and widely applicable. Experiment results clearly show the benefit of our method for both human activity parsing and prediction on three video datasets. [ABSTRACT FROM AUTHOR]

Generally, the internal states of most engineering systems are inaccessible to sensors under operational conditions, permitting indirect measurements and failures to be observed discretely. For this type of system, a class of proportional hazard (PH) models with multistate process have been proposed for performing prognostics. However, existing studies use a constant degradation mechanism with a very limited number of states, which may fail to model the actual deterioration of the system. Therefore, this article presents a novel PH model that utilizes a nonconstant degradation mechanism with a large number of states to perform health prognostics. A matrix partition-based approximation method is developed to include all the possible state transitions and to estimate important health quantities of the system, such as conditional reliability (CR), mean remaining useful life (MRUL), and remaining useful life distribution (RULD). An online prognostic scheme based on this matrix approach is presented to provide an update with the latest health quantities of the system. The proposed approach provides appealing computational features and avoids difficulties in calculating uncountable transitions for PH model-based prognostics. The effectiveness of the proposed method is demonstrated via a case of helical gearboxes under indirect monitoring and random failure conditions. [ABSTRACT FROM AUTHOR]

Human emotions unfold over time, and more affective computing research has to prioritize capturing this crucial component of real-world affect. Modeling dynamic emotional stimuli requires solving the twin challenges of time-series modeling and of collecting high-quality time-series datasets. We begin by assessing the state-of-the-art in time-series emotion recognition, and we review contemporary time-series approaches in affective computing, including discriminative and generative models. We then introduce the first version of the Stanford Emotional Narratives Dataset (SENDv1): a set of rich, multimodal videos of self-paced, unscripted emotional narratives, annotated for emotional valence over time. The complex narratives and naturalistic expressions in this dataset provide a challenging test for contemporary time-series emotion recognition models. We demonstrate several baseline and state-of-the-art modeling approaches on the SEND, including a Long Short-Term Memory model and a multimodal Variational Recurrent Neural Network, which perform comparably to the human-benchmark. We end by discussing the implications for future research in time-series affective computing. [ABSTRACT FROM AUTHOR]

Recommender systems are widely used in electronic commerce, social media and online streaming services to provide personalized recommendations to the users by exploiting past ratings and interactions. This paper considers the security aspect with quick and accurate detection of attacks by observing the newly created profiles sequentially to prevent the damage which may be incurred by the injection of new profiles with dishonest ratings. The proposed framework consists of a latent variable model, which is trained by a variational EM algorithm, followed by a sequential detection algorithm. The latent variable model generates homogeneous representations of the users given their rating history and mixed data-type attributes such as age and gender. The representations are then exploited to generate univariate statistics to be efficiently used in a CUSUM-like sequential detection algorithm that can quickly detect persistent attacks while maintaining low false alarm rates. We apply our proposed framework to three different real-world datasets and exhibit superior performance in comparison to the existing baseline algorithms for both attack profile and sequential detection. Furthermore, we demonstrate robustness to different attack strategies and configurations. [ABSTRACT FROM AUTHOR]

Accurate human mobility prediction is important for many applications in wireless networks, including intelligent content caching and prefetching, network optimization, etc. However, modeling mobility patterns has been a challenging problem due to the complicated human mobility patterns influenced by the long-term correlation with historical trajectories and context information, and the long time interval between consecutive mobility records. In this paper, we integrate the novel attention technique into the Markov model to predict future locations. This model allows us to consider the time context of users’ mobility behavior, which overcomes the disadvantage of the traditional Markov model in modeling the periodicity of trajectories. We conducted extensive evaluations using two different mobility datasets, which involve over 20,000 users. Our evaluations show that our proposed solution outperforms the state-of-the-art algorithms by over 6.6% on average. In addition, compared with the method based on deep recurrent neural networks achieving the same performance, our proposed model runs significantly faster, i.e., reducing the computation time by over 170 times, demonstrating the effeteness of our proposed model. [ABSTRACT FROM AUTHOR]

A visual scanpath represents the human eye movements when scanning the visual field for acquiring and receiving visual information. Predicting visual scanpaths when a certain stimulus is presented plays an important role in modeling overt human visual attention and search behavior. In this paper, we presented an ’Inhibition of Return - Region of Interest’ (IOR-ROI) recurrent mixture density network based framework learning to produce human-like visual scanpaths under task-free viewing conditions. The proposed model simultaneously predicts a sequence of ordered fixation positions and their corresponding fixation durations. Our model integrates bottom-up features and semantic features extracted by convolutional neural networks. Then the integrated feature maps are fed into the IOR-ROI Long Short-Term Memory (LSTM) which is the core component of the proposed model. The IOR-ROI LSTM is a dual LSTM unit, i.e., the IOR-LSTM and the ROI-LSTM, capturing IOR dynamics and gaze shift behavior simultaneously. IOR-LSTM simulates the visual working memory to adaptively maintain and update visual information regarding previously fixated regions. ROI-LSTM is responsible for predicting the next possible ROIs given the spatially inhibited image feature maps on the feature-wise basis. Fixation duration is predicted by a regression neural network given the viewing history and image feature maps corresponding to currently fixated ROI. Considering the eye movement pattern variations among subjects, a mixture density network is adopted to model the next fixation distribution as Gaussian mixtures and the fixation duration is also modeled using Gaussian distribution. Our model is evaluated on the OSIE and MIT low resolution eye-tracking datasets and experimental results indicate that the proposed method can achieve superior performance in predicting visual scanpaths. The code will be publicly available on URL: https://github.com/sunwj/scanpath . [ABSTRACT FROM AUTHOR]

The influence model is a discrete-time stochastic model that succinctly captures the interactions of a network of interacting Markov chains. The model produces a reduced-order representation of stochastic networks, and can be used to describe and tractably analyze probabilistic spatiotemporal spread dynamics, and hence has found broad usage in network applications, such as social networks, traffic management, and failure cascades in power systems. This article provides sufficient and necessary conditions for the identifiability of the influence model, and also develops estimators for model structures through exploiting the model’s special properties. In addition, we analyze conditions for the identifiability of the partially observed influence model (POIM), for which not all of the sites can be measured. We develop an expectation-maximization (EM) algorithm-based estimator for POIMs. [ABSTRACT FROM AUTHOR]

Gesture recognition becomes a thriving research area in modern human motion recognition systems. The intensification of demands on efficient interactive human–machine-interface systems, commercial objectives, and many other factors contributes to fuel this revival dynamics. Understanding human gestures becomes essential for prevention and health monitoring applications. In particular, analyzing hand gestures is of paramount importance in personalized healthcare-related applications to help practitioners providing more qualitative assessments of subject’s pathologies, such as Parkinson’s diseases. This work proposes a novel deep neural network approach to forecast future gestures from a given sequence of hand motion using a wearable capacitance sensor of an innovative gesture recognition hardware system. To do this, we use an attention-based recurrent neural network to capture the temporal features of hand motion to unveil the underlying pattern between the gesture and these sequences. While the attention layers capture patterns from the weights of the short term, the gated recurrent unit (GRU) neural network layer learns the inherent interdependency of long-term hand gesture temporal sequences. The efficiency of the proposed model is evaluated with respect to cutting-edge work in the field using several metrics. Note to Practitioners—In this article, the problem of human hand gesture recognition is analyzed using deep learning techniques. The proposed model uses input historical motion sequences collected from a wearable capacitance sensor to predict hand gestures. The model leverages the intrinsic correlation of motion sequences and extracts the salient part of the sequences by taking into consideration their temporal, complex, and nonlinear features. The approach studies the effect of different lengths of historical motion sequences in prediction outcomes. This allows for avoiding using cumbersome data collection, heavy data treatment, and high computational cost. The model performance is trained and assessed on real-world data by performing comparisons with alternative approaches, including well-known classifiers. The model yields very encouraging results and demonstrates that the proposed approach is quite competitive as it can reproduce typical activity trends for important channels. The present findings could help in the development of intelligent wearable devices for predicting hand gestures using a limited number of channels. This work could also help practitioners to provide a more qualitative appraisal of patients suffering from different pathologies such as Parkinson’s diseases to personalized healthcare-related applications and to develop wearable gesture recognition devices on a large scale. [ABSTRACT FROM AUTHOR]

Lebesgue-sampling-based fault diagnosis and prognosis (LS-FDP) is developed with advantages of light computation and small uncertainty accumulation. In previous research, the applications of LS-FDP are developed for single-model degradation processes, which do not consider degradation processes that involve multiple modes with switching models. To address this scenario, this article proposes a modified interacting multiple model filter in the Lebesgue-time–space-model-based FDP framework. The design, analysis, and particle filtering-based implementation of the proposed method are discussed in detail. A simulation example and a real experiment are presented to demonstrate the accuracy, effectiveness, robustness, and universality of the proposed method. [ABSTRACT FROM AUTHOR]