Search

The traditional method for transformer moisture diagnosis is to establish empirical equations between feature parameters extracted from frequency domain spectroscopy (FDS) and the transformer’s moisture content. However, the established empirical equation may not be applicable to a novel testing environment, resulting in an unreliable evaluation result. In this regard, it is acknowledged that FDS combined with machine learning is more suitable for estimating moisture content in a variety of test environments. Nonetheless, the accuracy of the estimation results obtained using the existing method is limited by the algorithm’s inability to generalize. To address this issue, we propose an AdaBoost algorithm-enhanced back-propagation neural network (BP_AdaBoost). This study creates a database by extracting feature parameters from the FDS that characterize the insulation states of the prepared samples. Then, using the BP_AdaBoost algorithm and the newly constructed database, the moisture estimation models are trained. Finally, the results of the estimation are discussed in terms of laboratory and field transformers. By comparing the proposed BP_AdaBoost algorithm to other intelligence algorithms, it is demonstrated that it not only performs better in generalization, but also maintains a high level of accuracy. [ABSTRACT FROM AUTHOR]

Name disambiguation is a key and also a very tough problem in many online systems such as social search and academic search. Despite considerable research, a critical issue that has not been systematically studied is disambiguation on the fly — to complete the disambiguation in the real-time. This is very challenging, as the disambiguation algorithm must be accurate, efficient, and error tolerance. In this paper, we propose a novel framework — CONNA — to train a matching component and a decision component jointly via reinforcement learning. The matching component is responsible for finding the top matched candidate for the given paper, and the decision component is responsible for deciding on assigning the top matched person or creating a new person. The two components are intertwined and can be bootstrapped via jointly training. Empirically, we evaluate CONNA on two name disambiguation datasets. Experimental results show that the proposed framework can achieve a 1.21-19.84 percent improvement on F1-score using joint training of the matching and the decision components. The proposed CONNA has been successfully deployed on AMiner — a large online academic search system. [ABSTRACT FROM AUTHOR]

Surface electromyographic signals (sEMG) usually have high-dimensional properties, and direct processing of these data consumes significant computational resources. Dimensionality reduction processing can reduce the dimension of the data and improve the real-time performance and response speed. This is especially important for application scenarios such as prosthetic control and rehabilitation training where rapid feedback is required. This paper proposes a feature fusion dimension reduction method for sEMG signals. This method is constructed based on the unique correlation between the features of sEMG. To test the performance of the new dimension reduction method, the sEMG signals from five leg movements were collected from eight subjects and the classification of the feature matrix before and after dimension reduction was tested by six classifiers. The results show that the feature matrix after fusion dimension reduction has excellent classification performance in the subsequent classification tasks. It produces up to 98.3% accuracy. And the highest comprehensive evaluation index can reach 0.9958. This paper also compares the new method with three commonly used dimensionality reduction methods. The results show that the performance of the new method is not only optimal but also extremely stable. Because its classification performance will not be lower than other dimensionality reduction methods due to the change of classifiers. This confirms that the new method has a higher utility value in sEMG signals processing compared to other dimension reduction methods. [ABSTRACT FROM AUTHOR]

Estimating depth from RGB images is a long-standing ill-posed problem, which has been explored for decades by the computer vision, graphics, and machine learning communities. Among the existing techniques, stereo matching remains one of the most widely used in the literature due to its strong connection to the human binocular system. Traditionally, stereo-based depth estimation has been addressed through matching hand-crafted features across multiple images. Despite the extensive amount of research, these traditional techniques still suffer in the presence of highly textured areas, large uniform regions, and occlusions. Motivated by their growing success in solving various 2D and 3D vision problems, deep learning for stereo-based depth estimation has attracted a growing interest from the community, with more than 150 papers published in this area between 2014 and 2019. This new generation of methods has demonstrated a significant leap in performance, enabling applications such as autonomous driving and augmented reality. In this paper, we provide a comprehensive survey of this new and continuously growing field of research, summarize the most commonly used pipelines, and discuss their benefits and limitations. In retrospect of what has been achieved so far, we also conjecture what the future may hold for deep learning-based stereo for depth estimation research. [ABSTRACT FROM AUTHOR]

Pneumatic artificial muscle (PAM) has been widely used in rehabilitation and other fields as a flexible and safe actuator. In this paper, a PAM-actuated wearable exoskeleton robot is developed for upper limb rehabilitation. However, accurate modeling and control of the PAM are difficult due to complex hysteresis. To solve this problem, this paper proposes an active neural network method for hysteresis compensation, where a neural network (NN) is utilized as the hysteresis compensator and unscented Kalman filtering is used to estimate the weights and approximation error of the NN in real time. Compared with other inversion-based methods, the NN is directly used as the hysteresis compensator without needing inversion. Additionally, the proposed method does not require pre-training of the NN since the weights can be dynamically updated. To verify the effectiveness and robustness of the proposed method, a series of experiments have been conducted on the self-built exoskeleton robot. Compared with other popular control methods, the proposed method can track the desired trajectory faster, and tracking accuracy is gradually improved through iterative learning and updating. [ABSTRACT FROM AUTHOR]

The existing surface electromyography-based pattern recognition system (sEMG-PRS) exhibits limited generalizability in practical applications. In this paper, we propose a stacked weighted random forest (SWRF) algorithm to enhance the long-term usability and user adaptability of sEMG-PRS. First, the weighted random forest (WRF) is proposed to address the issue of imbalanced performance in standard random forests (RF) caused by randomness in sampling and feature selection. Then, the stacking is employed to further enhance the generalizability of WRF. Specifically, RF is utilized as the base learner, while WRF serves as the meta-leaning layer algorithm. The SWRF is evaluated against classical classification algorithms in both online experiments and offline datasets. The offline experiments indicate that the SWRF achieves an average classification accuracy of 89.06%, outperforming RF, WRF, long short-term memory (LSTM), and support vector machine (SVM). The online experiments indicate that SWRF outperforms the aforementioned algorithms regarding long-term usability and user adaptability. We believe that our method has significant potential for practical application in sEMG-PRS. [ABSTRACT FROM AUTHOR]

This paper proposes a novel method for Virtual Metrology (VM) in plasma etch processes based on analysis of all time and wavelength samples of Optical Emission Spectroscopy (OES) signals. The new method flattens each OES signal into a single vector, after which Singular Value Decomposition (SVD) is performed on the matrix formed by vectors of flattened OES signals in the training dataset. Low rank SVD projections of flattened and standardized OES recordings served as inputs for Ridge Regression, Artificial Neural Network, and Random Forest based VM models. A VM study is then conducted on a dataset gathered from a major 300 mm wafer fabrication facility, showing that the use of newly proposed SVD-based OES features consistently outperformed benchmark VM model features. Additional analysis of feature importance performed based on the analytically tractable Ridge Regression VM model form demonstrated distinct time-frequency patterns of OES signal portions that were highly informative for prediction of relevant Critical Dimensions, clearly justifying the need to use the entire OES signals for VM. [ABSTRACT FROM AUTHOR]

Surface defect is a serious problem in real-world manufacturing system and it is important to use vision-based recognition to ensure the surface quality of products. Currently, due to the ability of automatic feature extraction, deep learning models, such as convolutional neural network (CNN), have been widely used in this area. However, these CNN-based models may not solve a problem well - inter-class similarities and intra-class variations (ISIV), which affect their ability of feature extraction and thus influence their recognition performance. To address this problem, this paper introduces a graph guidance mechanism into CNN to improve the ability of feature extraction, called Graph guided Convolutional Neural Network (GCNN). Firstly, GCNN defines a graph by computing the similarities between training samples. Secondly, the graph is introduced into VGG11, a popular CNN structure, to increase the inter-class distances and decrease the intra-class distances between defect samples. Meanwhile, a learnable coefficient is introduced into the training process to balance the effect of graph guidance automatically. The experimental results on four famous surface defect datasets demonstrate that the graph guidance helps CNN models have better ability of feature extraction and thus achieve better performance. Compared with state-of-the-art models, the proposed method can achieve the best performance. Furthermore, the final discussion shows that the learnable coefficient can help the proposed model to gain better performance, and that the proposed model increases a little computation cost compared to its original CNN model. Note to Practitioners—This paper is motivated by the problem in real-world manufacturing process – inter-class similarities and intra-class variations. Most of current CNN-based models may not solve this problem well, which limits their applications in real-world system. This paper proposes a graph guidance mechanism and introduces it to the training of CNN. The proposal can improve the ability of feature extraction, and thus have better performance than those without the graph guidance. By applying the proposal to four examples, the results show that the proposal is more feasible and effective than the state-of-the-art models in surface defect recognition. Furthermore, the mechanism assists the training of CNN and is removed in the process of recognition, so it will not increase time and occupied memory in recognition, and meanwhile, only limited computation cost is increased in training process. [ABSTRACT FROM AUTHOR]

Currently, federated reinforcement learning (FRL) is driving progress in the edge artificial intelligence (AI) sector by effectively addressing a range of environmental challenges through distributed learning. The selection of participants in federated learning (FL) is especially vital because it greatly influences the overall performance and learning efficiency of the system. However, unlike traditional FL, the absence of a pre-accumulated dataset in FRL makes current FL participant selection methods unsuitable for FRL systems. To address this limitation, this paper proposes a novel experience-based participant selection method, tailored for FRL systems. This approach uses dimensionality reduction techniques to effectively analyze the experiences of agents and employs similarity measurements to intelligently select agents with diverse experiences. The FRL system has been implemented using the proposed technique, and extensive experiments have been performed to assess the system's performance from various perspectives. Our evaluation results show a considerable increase in the intelligence of the FRL system, resulting in more efficient training and an overall enhancement in performance. This is demonstrated by a considerable reduction of 23.65% in average training time and an average performance improvement of 53.25% compared to existing methods. [ABSTRACT FROM AUTHOR]

Advancements in wearable sensor technologies and the digitization of medical records have contributed to the unprecedented ubiquity of biomedical time series data. Data-driven models have tremendous potential to assist clinical diagnosis and improve patient care by improving long-term monitoring capabilities, facilitating early disease detection and intervention, as well as promoting personalized healthcare delivery. However, accessing extensively labeled datasets to train data-hungry deep learning models encounters many barriers, such as long-tail distribution of rare diseases, cost of annotation, privacy and security concerns, data-sharing regulations, and ethical considerations. An emerging approach to overcome the scarcity of labeled data is to augment AI methods with human-like capabilities to leverage past experiences to learn new tasks with limited examples, called few-shot learning. This survey provides a comprehensive review and comparison of few-shot learning methods for biomedical time series applications. The clinical benefits and limitations of such methods are discussed in relation to traditional data-driven approaches. This paper aims to provide insights into the current landscape of few-shot learning for biomedical time series and its implications for future research and applications. [ABSTRACT FROM AUTHOR]

Increasing demands on medical imaging departments are taking a toll on the radiologist's ability to deliver timely and accurate reports. Recent technological advances in artificial intelligence have demonstrated great potential for automatic radiology report generation (ARRG), sparking an explosion of research. This survey paper conducts a methodological review of contemporary ARRG approaches by way of (i) assessing datasets based on characteristics, such as availability, size, and adoption rate, (ii) examining deep learning training methods, such as contrastive learning and reinforcement learning, (iii) exploring state-of-the-art model architectures, including variations of CNN and transformer models, (iv) outlining techniques integrating clinical knowledge through multimodal inputs and knowledge graphs, and (v) scrutinising current model evaluation techniques, including commonly applied NLP metrics and qualitative clinical reviews. Furthermore, the quantitative results of the reviewed models are analysed, where the top performing models are examined to seek further insights. Finally, potential new directions are highlighted, with the adoption of additional datasets from other radiological modalities and improved evaluation methods predicted as important areas of future development. [ABSTRACT FROM AUTHOR]

Machining conditions of real-time identification tools is a key and trending issue for the industry. This paper focuses on identifying whether machining is performed as well as the chatter conditions generated during re-machining processes. Identifying whether or not machining conditions are met allows users to ensure the normal operation of machining equipment and identify situations that do not match the current conditional, so that they can take early action and further save on operational costs for machining. The objective of this paper is to identify the milling machining conditions, and the identified conditions will be categorized into whether cutting is required as well as whether chatter is observed. In order to identify these three conditions, sound and vibration signals are captured by sensors inside the milling machine, and the process of identification is subsequently analyzed and conditions established. In this paper, in order to produce a valid model, the extracted machining signal is characterized as a training model by the properties of Approximate Entropy and Short-Time Fourier Transform, and the k-fold cross-validation criteria is utilized to present the identification results. Finally, In this study, the model recognition rate of support vector machine with approximate entropy was 91.4%. The recognition rate of the convolutional neural network with short time span Fourier transform was 95.5%. Finally, the reduced network architecture can significantly reduce the training time and maintain the recognition rate at 93.6%. [ABSTRACT FROM AUTHOR]

Heterogeneous graph embedding aims at learning low-dimensional representations from a graph featuring nodes and edges of diverse natures, and meanwhile preserving the underlying topology. Existing approaches along this line have largely relied on meta-paths, which are by nature hand-crafted and pre-defined transition rules, so as to explore the semantics of a graph. Despite the promising results, defining meta-paths requires domain knowledge, and thus when the test distribution deviates from the priors, such methods are prone to errors. In this paper, we propose a self-learning scheme for heterogeneous graph embedding, termed as self-guided walk (SILK), that bypasses meta-paths and learns adaptive attentions for node walking. SILK assumes no prior knowledge or annotation is provided, and conducts a customized random walk to encode the contexts of the heterogeneous graph of interest. Specifically, this is achieved via maintaining a dynamically-updated guidance matrix that records the node-conditioned transition potentials. Experimental results on four real-world datasets demonstrate that SILK significantly outperforms state-of-the-art methods. [ABSTRACT FROM AUTHOR]

A novel coronavirus disease (COVID-19) is a pandemic disease has caused 4 million deaths and more than 200 million infections worldwide (as of August 4, 2021). Rapid and accurate diagnosis of COVID-19 infection is critical to controlling the spread of the epidemic. In order to quickly and efficiently detect COVID-19 and reduce the threat of COVID-19 to human survival, we have firstly proposed a detection framework based on reinforcement learning for COVID-19 diagnosis, which constructs a mixed loss function that can integrate the advantages of multiple loss functions. This paper uses the accuracy of the validation set as the reward value, and obtains the initial model for the next epoch by searching the model corresponding to the maximum reward value in each epoch. We also have proposed a prediction framework that integrates multiple detection frameworks using parameter sharing to predict the progression of patients' disease without additional training. This paper also constructed a higher-quality version of the CT image dataset containing 247 cases screened by professional physicians, and obtained more excellent results on this dataset. Meanwhile, we used the other two COVID-19 datasets as external verifications, and still achieved a high accuracy rate without additional training. Finally, the experimental results show that our classification accuracy can reach 98.31%, and the precision, sensitivity, specificity, and AUC (Area Under Curve) are 98.82%, 97.99%, 98.67%, and 0.989, respectively. The accuracy of external verification can reach 93.34% and 91.05%. What's more, the accuracy of our prediction framework is 91.54%. A large number of experiments demonstrate that our proposed method is effective and robust for COVID-19 detection and prediction. [ABSTRACT FROM AUTHOR]

This paper serves as a survey of recent advances in large margin training and its theoretical foundations, mostly for (nonlinear) deep neural networks (DNNs) that are probably the most prominent machine learning models for large-scale data in the community over the past decade. We generalize the formulation of classification margins from classical research to latest DNNs, summarize theoretical connections between the margin, network generalization, and robustness, and introduce recent efforts in enlarging the margins for DNNs comprehensively. Since the viewpoint of different methods is discrepant, we categorize them into groups for ease of comparison and discussion in the paper. Hopefully, our discussions and overview inspire new research work in the community that aim to improve the performance of DNNs, and we also point to directions where the large margin principle can be verified to provide theoretical evidence why certain regularizations for DNNs function well in practice. We managed to shorten the paper such that the crucial spirit of large margin learning and related methods are better emphasized. [ABSTRACT FROM AUTHOR]

In this paper, a simple yet effective network pruning framework is proposed to simultaneously address the problems of pruning indicator, pruning ratio, and efficiency constraint. This paper argues that the pruning decision should depend on the convolutional weights, and thus proposes novel weight-dependent gates (W-Gates) to learn the information from filter weights and obtain binary gates to prune or keep the filters automatically. To prune the network under efficiency constraints, a switchable Efficiency Module is constructed to predict the hardware latency or FLOPs of candidate pruned networks. Combined with the proposed Efficiency Module, W-Gates can perform filter pruning in an efficiency-aware manner and achieve a compact network with a better accuracy-efficiency trade-off. We have demonstrated the effectiveness of the proposed method on ResNet34, ResNet50, and MobileNet V2, respectively achieving up to 1.33/1.28/1.1 higher Top-1 accuracy with lower hardware latency on ImageNet. Compared with state-of-the-art methods, W-Gates also achieves superior performance. [ABSTRACT FROM AUTHOR]

As an effective data augmentation method, Mixup synthesizes an extra amount of samples through linear interpolations. Despite its theoretical dependency on data properties, Mixup reportedly performs well as a regularizer and calibrator contributing reliable robustness and generalization to deep model training. In this paper, inspired by Universum Learning which uses out-of-class samples to assist the target tasks, we investigate Mixup from a largely under-explored perspective - the potential to generate in-domain samples that belong to none of the target classes, that is, universum. We find that in the framework of supervised contrastive learning, Mixup-induced universum can serve as surprisingly high-quality hard negatives, greatly relieving the need for large batch sizes in contrastive learning. With these findings, we propose Universum-inspired supervised Contrastive learning (UniCon), which incorporates Mixup strategy to generate Mixup-induced universum as universum negatives and pushes them apart from anchor samples of the target classes. We extend our method to the unsupervised setting, proposing Unsupervised Universum-inspired contrastive model (Un-Uni). Our approach not only improves Mixup with hard labels, but also innovates a novel measure to generate universum data. With a linear classifier on the learned representations, UniCon shows state-of-the-art performance on various datasets. Specially, UniCon achieves 81.7% top-1 accuracy on CIFAR-100, surpassing the state of art by a significant margin of 5.2% with a much smaller batch size, typically, 256 in UniCon vs. 1024 in SupCon (Khosla et al., 2020) using ResNet-50. Un-Uni also outperforms SOTA methods on CIFAR-100. The code of this paper is released on https://github.com/hannaiiyanggit/UniCon. [ABSTRACT FROM AUTHOR]

Parkinson’s disease (PD) is a neurodegenerative disease of the brain associated with motor symptoms. With the maturation of machine learning (ML), especially deep learning, ML has been used to assist in the diagnosis of PD. In this paper, we explore graph neural networks (GNNs) to implement PD prediction using MRI data. However, most existing GNN models suffer from the efficiency of graph construction on MRI data and the problem of overfitting on small data. This paper proposes a novel multi-layer GNN model that incorporates a fast graph construction method and a sparsity-based pooling layer with an attention mechanism. In addition, graph structure sparsity is plugged into the graph pooling layer as prior knowledge to mitigate overfitting in model training. Experimental results on real-world datasets demonstrate the effectiveness of the proposed model and its superiority over baseline methods. [ABSTRACT FROM AUTHOR]

The treatment of patients with balance disorders is an urgent problem to be solved by the medical community. The causes of balance disorders are diverse. An aging population, traffic accidents, stroke, genetic diseases and so on are all possible factors. It has brought great pain and inconvenience to patients and their families. At present, there are two main types of assisted rehabilitation training robots for patients with balance disorders: exoskeleton robots and end robots. The exoskeleton robot is generally installed on the outside of the patient’s body to follow their movement, which can support the weight of the body and provide power support to help the patient train and recover lower limb ability. The use of end robots is usually to secure the patient’s foot to the motion platform and control the pedal to drive the lower limbs to conduct gait training. Such passive training is more suitable for patients with severe disorders. The patient has low awareness of active participation. This paper focuses on research on end rehabilitation training robots for balance disorders. In this paper, a robotic system for rehabilitation training of patients with balance disorders is invented. The robot body is a 9 degree of freedom (DOF) redundant series-parallel hybrid motion platform. Two sets of motion platforms with symmetrical mirror images are used together to simulate different motion modes of the human body and drive the human body to move. Each set of motion platforms is composed of a 6-DOF vestibular parallel device and a 3-DOF proprioception parallel device. It has the advantages of DOF decoupling and fast response, proposing a new structural form for the design of proprioceptive and vestibular simulation platforms. The robot’s functional level can be divided into a vestibular sense module and a proprioception module according to the structure. The two modules can work independently to achieve different functions or work together to achieve complex motion and multisensory fusion. This robot is a redundant mechanism device with 9 DOFs. Through a reasonable distribution of DOF and motion, the robot’s working space can be increased, and the robot’s flexibility and motion performance can be improved. In this paper, a trajectory tracking control algorithm for vestibular and proprioceptive simulation is proposed, which can provide unlimited body sense training for patients within the robot’s limited motion range. [ABSTRACT FROM AUTHOR]

The semi-supervised face recognition problem has become a popular research topic in recent years. However, one common and important situation, in which the unlabeled data is shallow, has rarely been considered in most existing works. In this paper, shallow data means there are only few images per identity. In the unlabeled shallow situation, the existing semi-supervised face recognition methods generally do not work well. Thus, how to effectively utilize the unlabeled shallow face data for improving face recognition performance is an important issue. In this paper, we propose a novel semi-supervised face recognition method, namely VirFace $^{\infty} $ , to enhance the face recognition performance effectively with the unlabeled shallow data. VirFace $^{\infty} $ consists of VirClass and VirDistribution components. In VirClass, we inject the unlabeled data as virtual classes into the feature space to enlarge the inter-class distance. In VirDistribution, we predict the distribution of each virtual class, namely virtual distribution, and then enhance the inter-class discriminativeness by enlarging the distances between the labeled features and the virtual distributions. To the best of our knowledge, we are among the first to tackle the face recognition problem on unlabeled shallow face data. Extensive experiments demonstrate the superiority of our proposed method. [ABSTRACT FROM AUTHOR]

The fifth generation New Radio vehicle-to-everything (5G NR-V2X) has higher requirements for delay and reliability, which brings challenges to the physical layer signal processing. Mobile edge computing (MEC) can provide larger capacity data storage and more efficient computation through localization of on-board unit (OBU) and next generation Node B (gNB) services. This paper combines the channel estimation of New Radio vehicle-to-infrastructure (NR-V2I) communication baseband signal processing with MEC and designs an intelligent channel estimation framework. In the MEC server, this paper proposes a channel estimation algorithm based on deep learning. This algorithm uses a one-dimensional convolutional neural network (1D CNN) to complete frequency-domain interpolation and conditional recurrent unit (CRU) for time-domain state prediction. Additional velocity coding vector and multipath coding vector track changes in the environment, and accurately train channel data in different mobile environments. System simulation and analysis show that the proposed algorithm improves the channel estimation accuracy, reduces the bit error rate, and enhances the robustness compared with the representative channel estimation algorithms for NR-V2I. [ABSTRACT FROM AUTHOR]

An important aspect of querying whether a product is likely to be forged is to identify its anti-counterfeiting label. However, the use of image processing technology for label-specific texture analysis to quickly and effectively identify the anti-counterfeiting label has been widely studied. Aiming at the defects of the local binary pattern (LBP) and its variants in texture identification, this paper proposes a new texture model for anti-counterfeiting identification, that is, the circular local ternary pattern (CLTP). The highlight of our technology is that it extracts the effective local texture descriptors by using the random features of inkjet printing. This allows for the technology to not only resist the interference of noise and illumination in images of anti-counterfeiting patterns but also to encode and reorganize the fine linear shape structure. Specifically, this paper extracts the CLTP texture feature in the corresponding key areas and forms the final feature histogram vector for comparison through the one-to-one correspondence between the sample image and the inspected image of anti-counterfeiting pattern. Experiments prove that our method not only has high discrimination, stability and effectiveness but also provides a convenient and practical idea for anti-counterfeiting technology. [ABSTRACT FROM AUTHOR]

In this paper, we present a novel model for simultaneous stable co-saliency detection (CoSOD) and object co-segmentation (CoSEG). To detect co-saliency (segmentation) accurately, the core problem is to well model inter-image relations between an image group. Some methods design sophisticated modules, such as recurrent neural network (RNN), to address this problem. However, order-sensitive problem is the major drawback of RNN, which heavily affects the stability of proposed CoSOD (CoSEG) model. In this paper, inspired by RNN-based model, we first propose a multi-path stable recurrent unit (MSRU), containing dummy orders mechanisms (DOM) and recurrent unit (RU). Our proposed MSRU not only helps CoSOD (CoSEG) model captures robust inter-image relations, but also reduces order-sensitivity, resulting in a more stable inference and training process. Moreover, we design a cross-order contrastive loss (COCL) that can further address order-sensitive problem by pulling close the feature embedding generated from different input orders. We validate our model on five widely used CoSOD datasets (CoCA, CoSOD3k, Cosal2015, iCoseg and MSRC), and three widely used datasets (Internet, iCoseg and PASCAL-VOC) for object co-segmentation, the performance demonstrates the superiority of the proposed approach as compared to the state-of-the-art (SOTA) methods. [ABSTRACT FROM AUTHOR]

The explosive growth of Internet-of-Things (IoT) applications such as smart cities and Industry 4.0 have led to drastic increase in demand for wireless bandwidth, hence motivating the rapid development of new techniques for enhancing spectrum utilization needed by new generation wireless communication technologies. Among others, dynamic spectrum access (DSA) is one of the most widely accepted approaches. In this paper, as an enhancement of existing works, we take into consideration of inter-node collaborations in a dynamic spectrum environment. Typically, in such distributed circumstances, intelligent dynamic spectrum access almost invariably relies on self-learning to achieve dynamic spectrum access improvement. Whereas, this paper proposes a DSA scheme based on deep reinforcement learning to enhance spectrum and access efficiency. Unlike traditional Q-learning-based DSA, we introduce the following to enhance the spectrum efficiency in dynamic IoT spectrum environments. First, deep double Q-learning is adopted to perform local self-spectrum-learning for IoT terminals in order to achieve better dynamic access accuracy. Second, to accelerate learning convergence, federated learning (FL) in edge nodes is used to improve the self-learning. Third, multiple secondary users, who do not interfere with each other and have similar operation condition, are clustered for federated learning to enhance the efficiency of deep reinforcement learning. Comparing with the traditional distributed DSA with deep learning, the proposed scheme has faster access convergence speed due to the characteristic of global optimization for federated learning. Based on this, a framework of federated deep reinforcement learning (FDRL) for DSA is proposed. Furthermore, this scheme preserves privacy of IoT users in that FDRL only requires model parameters to be uploaded to edge servers. Simulations are performed to show the effectiveness of theproposed FDRL-based DSA framework. [ABSTRACT FROM AUTHOR]

Reconfigurable intelligent surfaces (RISs) have been recently considered as a promising candidate for energy-efficient solutions in future wireless networks. Their dynamic and low-power configuration enables coverage extension, massive connectivity, and low-latency communications. Due to a large number of unknown variables referring to the RIS unit elements and the transmitted signals, channel estimation and signal recovery in RIS-based systems are the ones of the most critical technical challenges. To address this problem, we focus on the RIS-assisted wireless communication system and present two joint channel estimation and signal recovery schemes based on message passing algorithms in this paper. Specifically, the proposed bidirectional scheme applies the Taylor series expansion and Gaussian approximation to simplify the sum-product procedure in the formulated problem. In addition, the inner iteration that adopts two variants of approximate message passing algorithms is incorporated to ensure robustness and convergence. Two ambiguities removal methods are also discussed in this paper. Our simulation results show that the proposed schemes show the superiority over the state-of-art benchmark method. We also provide insights on the impact of different RIS parameter settings on the proposed schemes. [ABSTRACT FROM AUTHOR]

Human-machine interface(HMI) technology has gradually become a research hotspot with the continuous development of computer technology and Internet of Things (IOT) technology. Hand gesture recognition (HGR) as an important part of HMI technology has been widely concerned. Among the many technical routes of HGR technology, the wearable HGR technology based on A-mode ultrasound shows great application potential due to its advantages such as lightweight device, free from sensors’ constraint etc. However, in the process of processing A-mode ultrasonic signal, the amplitude of the signal at the main position may vary due to muscle fatigue and strength etc. when the same gesture is repeated some time later. In this paper, we design a method to overcome this problem and accomplish HGR in offline state by setting the threshold and use normalization method for the energy feature of ultrasonic signal according to the threshold. Three machine learning algorithms, including support vector machine (SVM), linear discriminant analysis (LDA) and Naive Bayes (NB) were used to verify the feasibility of this method, and the average recognition accuracy in the experiment reached 95.26%. Signal stability is also verified to be improved. In addition, We design a recognition strategy based on NB algorithm in this paper so that the model can identify the unknown gestures (i.e. gestures that have not been trained by the model). Five known gestures and five unknown gestures were set in the experiment, the average recognition accuracy of the known gestures is 91.2%, and the unknown gestures is 81.8%. This method can be used to optimize the user experience of HGR system. [ABSTRACT FROM AUTHOR]

In semiconductor manufacturing, defect detection is pivotal for enhancing productivity and yield. This paper introduces a novel weakly supervised method, the Implicit Cross Domain Diffusion Model (ICDDM), designed to tackle defect pattern segmentation challenges in the absence of detailed pixel-wise annotations. ICDDM employs a generative model to estimate the joint distribution of images depicting defect patterns and background circuits, formulating this estimation as a Markov Chain and optimizing it through denoising score matching. Building on this, we propose the Cross Domain Latent Diffusion Model (CDLDM), inspired by the Latent Diffusion Model, which simplifies the diffusion process into a lower-dimensional latent space to boost detection efficiency. Further enhancing our model, we introduce the Knowledge Distillation Cross Domain Diffusion Model (KDCDDM), which utilizes CDLDM as a teacher model and a Generative Adversarial Network (GAN) as a student model. This approach significantly accelerates the diffusion process by reducing the number of necessary denoising iterations while maintaining robust model performance. This suite of techniques offers a comprehensive solution for efficient and effective defect detection in semiconductor production environments. [ABSTRACT FROM AUTHOR]

This paper proposes a deep learning framework to design distributed compression strategies in which distributed agents need to compress high-dimensional observations of a source, then send the compressed bits via bandwidth limited links to a fusion center for source reconstruction. Further, we require the compression strategy to be progressive so that it can adapt to the varying link bandwidths between the agents and the fusion center. Moreover, to ensure scalability, we investigate strategies that depend only on the local channel state information (CSI) at each agent. Toward this end, we use a data-driven approach in which the progressive linear combination and uniform quantization strategy at each agent are trained as a function of its local CSI. To deal with the challenges of modeling the quantization operations (which always produce zero gradients in the training of neural networks), we propose a novel approach of exploiting the statistics of the batch training data to set the dynamic ranges of the uniform quantizers. Numerically, we show that the proposed distributed estimation strategy designed with only local CSI can significantly reduce the signaling overhead and can achieve a lower mean-squared error distortion for source reconstruction than state-of-the-art designs that require global CSI at comparable overall communication cost. [ABSTRACT FROM AUTHOR]

Textures contain a wealth of image information and are widely used in various fields such as computer graphics and computer vision. With the development of machine learning, the texture synthesis and generation have been greatly improved. As a very common element in everyday life, wallpapers contain a wealth of texture information, making it difficult to annotate with a simple single label. Moreover, wallpaper designers spend significant time to create different styles of wallpaper. For this purpose, this paper proposes to describe wallpaper texture images by using multi-label semantics. Based on these labels and generative adversarial networks, we present a framework for perception driven wallpaper texture generation and style transfer. In this framework, a perceptual model is trained to recognize whether the wallpapers produced by the generator network are sufficiently realistic and have the attribute designated by given perceptual description; these multi-label semantic attributes are treated as condition variables to generate wallpaper images. The generated wallpaper images can be converted to those with well-known artist styles using CycleGAN. Finally, using the aesthetic evaluation method, the generated wallpaper images are quantitatively measured. The experimental results demonstrate that the proposed method can generate wallpaper textures conforming to human aesthetics and have artistic characteristics. [ABSTRACT FROM AUTHOR]

Training an accurate classifier for EEG-based brain-computer interface (BCI) requires EEG data from a large number of users, whereas protecting their data privacy is a critical consideration. Federated learning (FL) is a promising solution to this challenge. This paper proposes Federated classification with local Batch-specific batch normalization and Sharpness-aware minimization (FedBS) for privacy protection in EEG-based motor imagery (MI) classification. FedBS utilizes local batch-specific batch normalization to reduce data discrepancies among different clients, and sharpness-aware minimization optimizer in local training to improve model generalization. Experiments on three public MI datasets using three popular deep learning models demonstrated that FedBS outperformed six state-of-the-art FL approaches. Remarkably, it also outperformed centralized training, which does not consider privacy protection at all. In summary, FedBS protects user EEG data privacy, enabling multiple BCI users to participate in large-scale machine learning model training, which in turn improves the BCI decoding accuracy. [ABSTRACT FROM AUTHOR]

Decoding natural language from noninvasive brain signals has been an exciting topic with the potential to expand the applications of brain-computer interface (BCI) systems. However, current methods face limitations in decoding sentences from electroencephalography (EEG) signals. Improving decoding performance requires the development of a more effective encoder for the EEG modality. Nonetheless, learning generalizable EEG representations remains a challenge due to the relatively small scale of existing EEG datasets. In this paper, we propose enhancing the EEG encoder to improve subsequent decoding performance. Specifically, we introduce the discrete Conformer encoder (D-Conformer) to transform EEG signals into discrete representations and bootstrap the learning process by imposing EEG-language alignment from the early training stage. The D-Conformer captures both local and global patterns from EEG signals and discretizes the EEG representation, making the representation more resilient to variations, while early-stage EEG-language alignment mitigates the limitations of small EEG datasets and facilitates the learning of the semantic representations from EEG signals. These enhancements result in improved EEG representations and decoding performance. We conducted extensive experiments and ablation studies to thoroughly evaluate the proposed method. Utilizing the D-Conformer encoder and bootstrapping training strategy, our approach demonstrates superior decoding performance across various tasks, including word-level, sentence-level, and sentiment-level decoding from EEG signals. Specifically, in word-level classification, we show that our encoding method produces more distinctive representations and higher classification performance compared to the EEG encoders from existing methods. At the sentence level, our model outperformed the baseline by 5.45%, achieving a BLEU-1 score of 42.31%. Furthermore, in sentiment classification, our model exceeded the baseline by 14%, achieving a sentiment classification accuracy of 69.3%. [ABSTRACT FROM AUTHOR]

Stroke survivors usually exhibit concurrent motor and cognitive impairment. Historically, rehabilitation strategies post-stroke occur separately in terms of motor and cognitive functions. However, recent studies show that hand motor interventions can have a positive impact on cognitive recovery. In this work, we introduce AMBER (portAble and Modular device for comprehensive Brain Evaluation and Rehabilitation), a new device developed for the evaluation and rehabilitation of both hand motor function and cognition simultaneously. AMBER is a simple, portable, ergonomic and cheap device based on Force Sensitive Resistors, in which every finger interaction is recorded to provide information about finger strength, processing speed, and memory status. This paper presents the requirements of the device and the design of the system. In addition, a pilot study was conducted with 36 healthy individuals using the evaluation module of the device to assess its psychometric properties, as test-retest reliability and measurement error. Its validity was also evaluated comparing its measurements with three different gold standards for strength, processing speed and memory. The device showed good test-retest reliability for strength (ICC =0.741-0.852), reaction time (ICC =0.715 – 0.900) and memory (ICC =0.556-0.885). These measures were correlated with their corresponding gold standards (r =0.780-890). AMBER shows great potential to impact hand rehabilitation, offering therapists a valid, reliable and versatile tool to comprehensively assess patients. With ongoing advancements and refinements, it has the opportunity to significantly impact rehabilitation practices and improve patient outcomes. [ABSTRACT FROM AUTHOR]

Motor imagery (MI) based brain-computer interfaces (BCIs) enable the direct control of external devices through the imagined movements of various body parts. Unlike previous systems that used fixed-length EEG trials for MI decoding, asynchronous BCIs aim to detect the user’s MI without explicit triggers. They are challenging to implement, because the algorithm needs to first distinguish between resting-states and MI trials, and then classify the MI trials into the correct task, all without any triggers. This paper proposes a sliding window prescreening and classification (SWPC) approach for MI-based asynchronous BCIs, which consists of two modules: a prescreening module to screen MI trials out of the resting-state, and a classification module for MI classification. Both modules are trained with supervised learning followed by self-supervised learning, which refines the feature extractors. Within-subject and cross-subject asynchronous MI classifications on four different EEG datasets validated the effectiveness of SWPC, i.e., it always achieved the highest average classification accuracy, and outperformed the best state-of-the-art baseline on each dataset by about 2%. [ABSTRACT FROM AUTHOR]

Speaker recognition systems (SRSs) are commonly used for biometric identification. However, these systems are vulnerable to adversarial attacks. Several defenses have been proposed but they require high costs in terms of additional data and computational resources to ensure robustness. To address these issues, this paper proposes a low-cost input reconstruction defense method called adaptive F-ratio-based partial masking (AFPM), which utilizes a robust feature extraction process to guarantee high defensibility. The underlying distribution of non-robust features is explored and filtered out by partial masking (PM), which helps maintain a low defense construction cost. An F-ratio-based PM (FPM) defense strategy is proposed by integrating the F-ratio, which reflects the weight of each frequency band for distinguishing between speakers, to balance classification accuracy and defensiveness. AFPM, which introduces an adaptive threshold calculation algorithm to FPM, is proposed to achieve further improved defensiveness and flexibility. Comparative experimental results show that AFPM is low-cost, highly defensive and universal. The construction process of AFPM does not involve training and its implementation does not require the protected SRSs to be retrained, only fine-tuned. While maintaining the classification accuracy at 99.42%, the average defense capability of AFPM against five white-box adaptive attacks is 90.89%, which is 9.23% better than that of the low-cost input reconstruction defense method and 3.77% better than that of the high-cost Parallel WaveGAN (PWG) defense approach. Against grey- and black-box adaptive attacks, FAKEBOB and Kenansville, AFPM reaches maximum defense effects of 96.01% and 74.49%, respectively, surpassing PWG by 4.5% and 65.82%. Furthermore, AFPM is universal and capable of protecting various SRSs against different attack strengths. [ABSTRACT FROM AUTHOR]

This paper presents an asset health index (HI) prediction methodology for high voltage transmission overhead lines (OHLs) using supervised machine learning and structured, unambiguous visual inspections. We propose a framework for asset HI predictions to determine the technical condition of individual OHL towers to improve grid reliability in a cost-effective manner. The paper focuses on asset HI prediction and the selection of the most parsimonious model. Based on the technical specifications and HI data, our methodology allows for the prediction of a HI for OHLs without HI data, and models asset aging behaviour. Technical specifications and the HI as defined in this paper are taken from the Estonian TSO periodical visual inspections implemented in 2018. The case study successfully demonstrates that the proposed methodology can predict tower HI values for a single OHL with nearly 80 percent accuracy without the need for additional measurements. [ABSTRACT FROM AUTHOR]

Accurate retinal fluid segmentation on Optical Coherence Tomography (OCT) images plays an important role in diagnosing and treating various eye diseases. The art deep models have shown promising performance on OCT image segmentation given pixel-wise annotated training data. However, the learned model will achieve poor performance on OCT images that are obtained from different devices (domains) due to the domain shift issue. This problem largely limits the real-world application of OCT image segmentation since the types of devices usually are different in each hospital. In this paper, we study the task of cross-domain OCT fluid segmentation, where we are given a labeled dataset of the source device (domain) and an unlabeled dataset of the target device (domain). The goal is to learn a model that can perform well on the target domain. To solve this problem, in this paper, we propose a novel Structure-guided Cross-Attention Network (SCAN), which leverages the retinal layer structure to facilitate domain alignment. Our SCAN is inspired by the fact that the retinal layer structure is robust to domains and can reflect regions that are important to fluid segmentation. In light of this, we build our SCAN in a multi-task manner by jointly learning the retinal structure prediction and fluid segmentation. To exploit the mutual benefit between layer structure and fluid segmentation, we further introduce a cross-attention module to measure the correlation between the layer-specific feature and the fluid-specific feature encouraging the model to concentrate on highly relative regions during domain alignment. Moreover, an adaptation difficulty map is evaluated based on the retinal structure predictions from different domains, which enforces the model focus on hard regions during structure-aware adversarial learning. Extensive experiments on the three domains of the RETOUCH dataset demonstrate the effectiveness of the proposed method and show that our approach produces state-of-the-art performance on cross-domain OCT fluid segmentation. [ABSTRACT FROM AUTHOR]

For patients who need lower-limb kinetism rehabilitation training, this paper proposes an effective data-driven approach seeking the design of 1-degree-of-freedom (DOF) six-bar rehab mechanism through gait prediction by body parameters. First, gait trajectories from 79 healthy volunteers are collected along with their body parameters. Then, the normalized gait samples are clustered and regressed into a limited number of representative trajectories with K-means algorithm, and the cluster index is recorded as the label for each trajectory. Next, a genetic-algorithm-optimized support vector machine method is adopted to establish a classifier for the trajectories, obtaining the correspondence between body parameters and cluster labels of gait trajectories. As a result, once a group of body parameters are input into the classifier, the suitable gait trajectory can be predicted for the specific patient. A GA-BFGS algorithm is developed for 1-DOF six-bar mechanism synthesis and a GUI design software is presented that shows how the data-driven design process is realized. The novelty of this paper is using clustering and prediction technique to accomplish the patient-mechanism matching, so that simple, low-priced 1-DOF mechanisms could be adopted for large number of various patients without expensive customized design for each individual. In the end, a gait rehab device design example is provided, and a prototype device driven by a constant speed motor is presented, which illustrates the feasibility of the proposed method. [ABSTRACT FROM AUTHOR]

In the field of reversible data hiding (RDH), how to predict an image and embed a message into the image with smaller distortion are two important aspects. In this paper, we propose a novel and efficient RDH method by innovating an intelligent predictor and an adaptive embedding way. In the prediction stage, we first constructed a convolutional neural network (CNN) based predictor by reasonably dividing an image into four parts. In such a way, each part can be predicted by using the other three parts as the context for the improvement of the prediction performance. Compared with existing predictors, the proposed CNN predictor can use more neighboring pixels for the prediction by exploiting its multi-receptive fields and global optimization capacities. In the embedding stage, we also developed a prediction-error-ordering (PEO) based adaptive embedding strategy, which can better adapt image content and thus efficiently reduce the embedding distortion by elaborately and luminously applying background complexity to select and pair those smaller prediction errors for data hiding. With the proposed CNN prediction and embedding ways, the RDH method presented in this paper provides satisfactory results in improving the visual quality of data hidden images, e.g., the average PSNR value for the Kodak benchmark dataset can reach as high as 63.59 dB with an embedding capacity of 10,000 bits. Extensive experimental results have shown that the RDH method proposed in this paper is superior to those existing state-of-the-art works. [ABSTRACT FROM AUTHOR]

Deep learning models have been shown to be vulnerable to adversarial attacks. Adversarial attacks are imperceptible perturbations added to an image such that the deep learning model misclassifies the image with a high confidence. Existing adversarial defenses validate their performance using only the classification accuracy. However, classification accuracy by itself is not a reliable metric to determine if the resulting image is “adversarial-free”. This is a foundational problem for online image recognition applications where the ground-truth of the incoming image is not known and hence we cannot compute the accuracy of the classifier or validate if the image is “adversarial-free” or not. This paper proposes a novel privacy preserving framework for defending Black box classifiers from adversarial attacks using an ensemble of iterative adversarial image purifiers whose performance is continuously validated in a loop using Bayesian uncertainties. The proposed approach can convert a single-step black box adversarial defense into an iterative defense and proposes three novel privacy preserving Knowledge Distillation (KD) approaches that use prior meta-information from various datasets to mimic the performance of the Black box classifier. Additionally, this paper proves the existence of an optimal distribution for the purified images that can reach a theoretical lower bound, beyond which the image can no longer be purified. Experimental results on six public benchmark datasets namely: 1) Fashion-MNIST, 2) CIFAR-10, 3) GTSRB, 4) MIO-TCD, 5) Tiny-ImageNet, and 6) MS-Celeb show that the proposed approach can consistently detect adversarial examples and purify or reject them against a variety of adversarial attacks. [ABSTRACT FROM AUTHOR]

Multi-Task Learning (MTL) is a learning paradigm in machine learning and its aim is to leverage useful information contained in multiple related tasks to help improve the generalization performance of all the tasks. In this paper, we give a survey for MTL from the perspective of algorithmic modeling, applications and theoretical analyses. For algorithmic modeling, we give a definition of MTL and then classify different MTL algorithms into five categories, including feature learning approach, low-rank approach, task clustering approach, task relation learning approach and decomposition approach as well as discussing the characteristics of each approach. In order to improve the performance of learning tasks further, MTL can be combined with other learning paradigms including semi-supervised learning, active learning, unsupervised learning, reinforcement learning, multi-view learning and graphical models. When the number of tasks is large or the data dimensionality is high, we review online, parallel and distributed MTL models as well as dimensionality reduction and feature hashing to reveal their computational and storage advantages. Many real-world applications use MTL to boost their performance and we review representative works in this paper. Finally, we present theoretical analyses and discuss several future directions for MTL. [ABSTRACT FROM AUTHOR]

Heart sound auscultation has been applied in clinical usage for early screening of cardiovascular diseases. Due to the high demand for auscultation expertise, automatic auscultation can help with auxiliary diagnosis and reduce the burden of training professional clinicians. Nevertheless, there is a limit to classic machine learning's performance improvement in the era of Big Data. Deep learning has outperformed classic machine learning in many research fields, as it employs more complex model architectures with a stronger capability of extracting effective representations. Moreover, it has been successfully applied to heart sound analysis in the past years. As most review works about heart sound analysis were carried out before 2017, the present survey is the first to work on a comprehensive overview to summarise papers on heart sound analysis with deep learning published in 2017–2022. This work introduces both classic machine learning and deep learning for comparison, and further offer insights about the advances and future research directions in deep learning for heart sound analysis. [ABSTRACT FROM AUTHOR]

Detecting wafer map anomalies is crucial for preventing yield loss in semiconductor fabrication, although intricate patterns and resource-intensive labeled data prerequisites hinder precise deep-learning segmentation. This paper presents an innovative, unsupervised method for detecting pixel-level anomalies in wafer maps. It utilizes an efficient dual attention module with a knowledge distillation network to learn defect distributions without anomalies. Knowledge transfer is achieved by distilling information from a pre-trained teacher into a student network with similar architecture, except an efficient dual attention module is incorporated atop the teacher network’s feature pyramid hierarchies, which enhances feature representation and segmentation across pyramid hierarchies that selectively emphasize relevant and discard irrelevant features by capturing contextual associations in positional and channel dimensions. Furthermore, it enables student networks to acquire an improved knowledge of hierarchical features to identify anomalies across different scales accurately. The dissimilarity in feature pyramids acts as a discriminatory function, predicting the likelihood of an abnormality, resulting in highly accurate pixel-level anomaly detection. Consequently, our proposed method excelled on the WM-811K and MixedWM38 datasets, achieving AUROC, AUPR, AUPRO, and F1-Scores of (99.65%, 99.35%), (97.31%, 92.13%), (90.76%, 84.66%) respectively, alongside an inference speed of 3.204 FPS, showcasing its high precision and efficiency. [ABSTRACT FROM AUTHOR]

It is known that the determination of the good-dice-in-bad-neighborhoods (GDBNs) has been regarded as an effective technique to reduce the value of the defect parts per million (DPPM) by identifying and rejecting the suspicious dice even though they are good in testing. Instead of examining eight immediate neighbors in a small-sized $3\times 3$ window or exploiting simple linear regression, a large-sized window can be used to recognize the broad-sighted neighborhoods and accurately infer the suspiciousness level for any given die. In this paper, the artificial neural networks (ANN)-based method can be proposed to solve the GDBN identification. Furthermore, two enhanced techniques can be further presented to improve the inference accuracy of the original ANN-based method by considering the variation of the time-dependent wafer patterns and the wafer-to-wafer relationship between two adjacent wafers. After applying the two enhanced techniques, the business profits can be improved in the new ANN-based method. Various experiments on two datasets clearly reveal the superiority of the proposed ANN-based method over the other existing methods. In addition to the reduction of the DPPM value, the new ANN-based method can achieve the 1.5X–2X better reduction in the cost of the return merchandise authorization (RMA). On the other hand, the experimental results show that the similar result can also be obtained in the other lower-yield products. By using the new ANN-based method, the relationships on bad dice cross wafers can be captured and the highly-accurate inference results can be simultaneously maintained. [ABSTRACT FROM AUTHOR]

This article presents a paradigm that adapts general large-scale pretrained models (PTMs) to speech emotion recognition task. Although PTMs shed new light on artificial general intelligence, they are constructed with general tasks in mind, and thus, their efficacy for specific tasks can be further improved. Additionally, employing PTMs in practical applications can be challenging due to their considerable size. Above limitations spawn another research direction, namely, optimizing large-scale PTMs for specific tasks to generate task-specific PTMs that are both compact and effective. In this paper, we focus on the speech emotion recognition task and propose an improVed emotion-specific pretrained encoder called Vesper. Vesper is pretrained on a speech dataset based on WavLM and takes into account emotional characteristics. To enhance sensitivity to emotional information, Vesper employs an emotion-guided masking strategy to identify the regions that need masking. Subsequently, Vesper employs hierarchical and cross-layer self-supervision to improve its ability to capture acoustic and semantic representations, both of which are crucial for emotion recognition. Experimental results on the IEMOCAP, MELD, and CREMA-D datasets demonstrate that Vesper with 4 layers outperforms WavLM Base with 12 layers, and the performance of Vesper with 12 layers surpasses that of WavLM Large with 24 layers. [ABSTRACT FROM AUTHOR]

Conventional continuous emotion prediction systems are typically trained to predict the ‘average’ of affect ratings obtained from multiple human annotators. These systems, however, ignore the ambiguity inherent in the perceived emotions, which is not captured by the ‘average rating’. This paper presents a novel ambiguity-aware continuous emotion prediction system that predicts the time-varying emotion state as a series of beta distributions. Our recent work has shown beta distributions to be an effective parametric model of a collection of affect ratings. This work develops an appropriate cost function that enables neural networks to be trained to predict beta distributions. It also investigates the choice of parameterization of the beta distribution, the choice of activation functions of the output layer, and the tractability of gradient definitions in combination with the loss function. The proposed framework is implemented using a Bag-of-Audio-Words front-end and an LSTM-based back-end and evaluated on the RECOLA dataset. In addition to comparison with baseline systems that only predict the ‘average rating’, the effectiveness with which the predictions represent ambiguity in perceived emotions is also evaluated. Experimental results reveal that the proposed approach outperforms other ambiguity-aware systems, especially when predicting valence. [ABSTRACT FROM AUTHOR]

Multi-modal classifiers for emotion recognition have become prominent, as the emotional states of subjects can be more comprehensively inferred from Electroencephalogram (EEG) signals and eye movements. However, existing classifiers experience a decrease in performance due to the distribution shift when applied to new users. Unsupervised domain adaptation (UDA) emerges as a solution to address the distribution shift between subjects by learning a shared latent feature space. Nevertheless, most UDA approaches focus on a single modality, while existing multi-modal approaches do not consider that fine-grained structures should also be explicitly aligned and the learned feature space must be discriminative. In this paper, we propose Coarse and Fine-grained Distribution Alignment with Correlated and Separable Features (CFDA-CSF), which performs a coarse alignment over the global feature space, and a fine-grained alignment between modalities from each domain distribution. At the same time, the model learns intra-domain correlated features, while a separable feature space is encouraged on new subjects. We conduct an extensive experimental study across the available sessions on three public datasets for multi-modal emotion recognition: SEED, SEED-IV, and SEED-V. Our proposal effectively improves the recognition performance in every session, achieving an average accuracy of 93.05%, 85.87% and 91.20% for SEED; 85.72%, 89.60%, and 86.88% for SEED-IV; and 88.49%, 91.37% and 91.57% for SEED-V. [ABSTRACT FROM AUTHOR]

The success of supervised models for emotion recognition on images heavily depends on the availability of images properly annotated. Although millions of images are presently available, only a few are annotated with reliable emotional information. Current emotion recognition solutions either use large amounts of weakly-labeled web images, which often contain noise that is unrelated to the emotions of the image, or transfer learning, which usually results in performance losses. Thus, it would be desirable to know which images would be useful to be annotated to avoid an extensive annotation effort. In this paper, we propose a novel approach based on active learning to choose which images are more relevant to be annotated. Our approach dynamically combines multiple active learning strategies and learns the best ones (without prior knowledge of the best ones). Experiments using nine benchmark datasets revealed that: (i) active learning allows to reduce the annotation effort, while reaching or surpassing the performance of a supervised baseline with as little as 3% to 18% of the baseline's training set, in classification tasks; (ii) our online combination of multiple strategies converges to the performance of the best individual strategies, while avoiding the experimentation overhead needed to identify them. [ABSTRACT FROM AUTHOR]

In this paper, we describe a general-purpose dialogue management framework used to design SOPHIE (Standardized Online Patient for Healthcare Interaction Education). SOPHIE simulates a virtual standardized cancer patient that allows physicians to practice skills such as empathy and patient empowerment in end-of-life communication. To provide the user with an opportunity to practice these skills, SOPHIE must produce a natural, emotionally appropriate conversation, yet handle topic shifts and open-ended questions from the user. To accomplish this, our approach to dialogue management loosely follows schemas – explicit representations of the typical flows of dialogue in end-of-life communication – while also using flexible pattern-driven methods for interpretation and generation. We conduct a crowdsourced evaluation of conversations between medical students and SOPHIE. Our agent is judged to produce responses that are natural, emotionally appropriate, and consistent with her role as a cancer patient. Furthermore, it significantly outperforms an end-to-end neural model fine-tuned on a human standardized patient corpus, attesting to the advantages of a schema-guided approach in this domain. However, the system is currently limited in its ability to generate responses that are judged to demonstrate deep understanding of the user, suggesting that future work should place focus on integrating this framework with robust natural language understanding and commonsense reasoning methods. [ABSTRACT FROM AUTHOR]

The ability to perceive emotions is an important criterion for judging whether a machine is intelligent. To this end, a large number of emotion recognition algorithms have been developed especially for visual information such as video. Most previous studies are based on hand-crafted features or CNN, in which the former fails to extract expressive features and the latter still faces the undesired affective gap. This drives us to think about what if we could incorporate the human emotional perception capability into CNN. In this paper, we attempt to address this question by exploring alignment between representations of neural networks and human brain activity. In particular, we employ a visually evoked emotional brain activity dataset to conduct a jointly training strategy for CNN. In the training phase, we introduce the representation similarity analysis (RSA) to align the CNN with human brain to obtain more brain-like features. Specifically, representation similarity matrices (RSMs) of multiple convolutional layers are averaged with learnable weights and related to the RSM of human brain. In order to obtain emotion-related brain activity, we conduct voxel selection and denoising with a banded ridge model before computing the RSM. Sufficient experiments on two challenging video emotion recognition datasets and multiple popular CNN architectures suggest that human brain activity is promising to provide an inductive bias for CNN towards better performance of emotion recognition. [ABSTRACT FROM AUTHOR]

This paper introduces video domain generalization where most video classification networks degenerate due to the lack of exposure to the target domains of divergent distributions. We observe that the global temporal features are less generalizable, due to the temporal domain shift that videos from other unseen domains may have an unexpected absence or misalignment of the temporal relations. This finding has motivated us to solve video domain generalization by effectively learning the local-relation features of different timescales that are more generalizable, and exploiting them along with the global-relation features to maintain the discriminability. This paper presents the VideoDG framework with two technical contributions. The first is a new deep architecture named the Adversarial Pyramid Network, which improves the generalizability of video features by capturing the local-relation, global-relation, and cross-relation features progressively. On the basis of pyramid features, the second contribution is a new and robust approach of adversarial data augmentation that can bridge different video domains by improving the diversity and quality of augmented data. We construct three video domain generalization benchmarks in which domains are divided according to different datasets, different consequences of actions, or different camera views, respectively. VideoDG consistently outperforms the combinations of previous video classification models and existing domain generalization methods on all benchmarks. [ABSTRACT FROM AUTHOR]