Search

Forgery of paper invoices and certificates negatively causes huge economic loss every year. In the molding process of paper, plant fibres inside will distribute randomly and form the unique microscopic surface. Accordingly, forensic researchers have presented many techniques to build its feature descriptor for paper anti-counterfeiting. Among those methods, normal vector field and reconstructed surface based on multi-view photos captured by mobile camera have indicated incredibly effectiveness and practicality. However, the inefficiency of existing models using mobile camera is the neck in real time detection scenarios. In this paper, we focus on the efficient microstructure orientation estimation of paper surface for authentication. To address this problem, we investigate the reflection characteristics of rough surface and quantify the light attenuation accurately. Then, we model the entire propagation path of light emitted by flash, and exploit single photo captured by mobile camera to reconstruct microscopic inclination based on beam energy loss. Reduction of required pictures significantly enhances forensics efficiency. Through comparison experiments, we demonstrate that our proposed method can be used for paper discrimination effectively with speedy feature extraction.

The study of keyword spotting, a subfield within the broader field of speech recognition that centers around identifying individual keywords in speech audio, has gained particular importance in recent years with the rise of personal voice assistants such as Alexa. As voice assistants aim to rapidly expand to support new languages, keywords, and use cases, stakeholders face the issue of limited training data for these unseen scenarios. This paper details some initial exploration into the application of Text-To-Speech (TTS) audio as a "helper" tool for training keyword spotters in these low-resource scenarios. In the experiments studied in this paper, the careful mixing of TTS audio with human speech audio during training led to a reduction of over 11% in the detection-error-tradeoff (DET) area under the curve (AUC) metric.

Hyperdimensional (HD) computing holds promise for classifying two groups of data. This paper explores seizure detection from electroencephalogram (EEG) from subjects with epilepsy using HD computing based on power spectral density (PSD) features. Publicly available intra-cranial EEG (iEEG) data collected from 4 dogs and 8 human patients in the Kaggle seizure detection contest are used in this paper. This paper explores two methods for classification. First, few ranked PSD features from small number of channels from a prior classification are used in the context of HD classification. Second, all PSD features extracted from all channels are used as features for HD classification. It is shown that for about half the subjects small number features outperform all features in the context of HD classification, and for the other half, all features outperform small number of features. HD classification achieves above 95% accuracy for six of the 12 subjects, and between 85-95% accuracy for 4 subjects. For two subjects, the classification accuracy using HD computing is not as good as classical approaches such as support vector machine classifiers.

Syntactic structure of a sentence text is correlated with the prosodic structure of the speech that is crucial for improving the prosody and naturalness of a text-to-speech (TTS) system. Nowadays TTS systems usually try to incorporate syntactic structure information with manually designed features based on expert knowledge. In this paper, we propose a syntactic representation learning method based on syntactic parse tree traversal to automatically utilize the syntactic structure information. Two constituent label sequences are linearized through left-first and right-first traversals from constituent parse tree. Syntactic representations are then extracted at word level from each constituent label sequence by a corresponding uni-directional gated recurrent unit (GRU) network. Meanwhile, nuclear-norm maximization loss is introduced to enhance the discriminability and diversity of the embeddings of constituent labels. Upsampled syntactic representations and phoneme embeddings are concatenated to serve as the encoder input of Tacotron2. Experimental results demonstrate the effectiveness of our proposed approach, with mean opinion score (MOS) increasing from 3.70 to 3.82 and ABX preference exceeding by 17% compared with the baseline. In addition, for sentences with multiple syntactic parse trees, prosodic differences can be clearly perceived from the synthesized speeches., Comment: This paper was submitted to ICASSP2021

Photo Response Non-Uniformity (PRNU) noise obtained from images or videos is used as a camera fingerprint to attribute visual objects captured by a camera. The PRNU-based source attribution method, however, fails when there is misalignment between the fingerprint and the query object. One example of such a misalignment, which has been overlooked in the field, is caused by the in-camera resizing technique that a video may have been subjected to. This paper investigates the attribution of visual media in the context of matching a video query object to an image fingerprint or vice versa. Specifically this paper focuses on improving camera attribution performance by taking into account the effects of binning, a commonly used in-camera resizing technique applied to video. We experimentally show that the True Positive Rate (TPR) obtained when binning is considered is approximately 3% higher.

This paper concerns the supervised generative non parametric autoencoder. Classical methods are based on variational non supervised autoencoders (VAE). Variational autoencoders encourage the latent space to fit a prior distribution, like a Gaussian. However, they tend to draw stronger assumptions for the data, often leading to higher asymptotic bias when the model is wrong.In this paper, we relax the parametric distribution assumption in the latent space and we propose to learn a non-parametric data distribution of the clusters in the latent space. The network encourages the latent space to fit a distribution learned with the labels instead of the parametric prior assumptions. We have built a network architecture that uses the labels to compute the latent space. Thus we define a global criterion combining classification and reconstruction loss. In addition, we have proposed a l 1,1 regularization which has the advantage of sparsifying the network and improving the clustering. Finally we propose a tailored algorithm to minimize the criterion with constraint. We demonstrate the effectiveness of our method using the popular image dataset MNIST and two biological datasets.

The paper is motivated by the importance of the Smart Cities (SC) concept for future management of global urbanization and energy consumption. Multi-agent Reinforcement Learning (RL) is an efficient solution to utilize large amount of sensory data provided by the Internet of Things (IoT) infrastructure of the SCs for city-wide decision making and managing demand response. Conventional ModelFree (MF) and Model-Based (MB) RL algorithms, however, use a fixed reward model to learn the value function rendering their application challenging for ever changing SC environments. Successor Representations (SR)-based techniques are attractive alternatives that address this issue by learning the expected discounted future state occupancy, referred to as the SR, and the immediate reward of each state. SR-based approaches are, however, mainly developed for single agent scenarios and have not yet been extended to multi-agent settings. The paper addresses this gap and proposes the Multi-Agent Adaptive Kalman Filtering-based Successor Representation (MAKF-SR) framework. The proposed framework can adapt quickly to the changes in a multi-agent environment faster than the MF methods and with a lower computational cost compared to MB algorithms. The proposed MAKF-SR is evaluated through a comprehensive set of experiments illustrating superior performance compared to its counterparts.

International audience; In this paper, we assume a set of mobile geolocalized sensor arrays observing an area over time. Each of these arrays consists of heterogeneous and cross-sensitive sensors, i.e., the sensor readings provided by one of such sensors also depends on the readings of the other sensors in the array. We further assume that such arrays are possibly-uncalibrated and we aim to propose an in situ calibration method---i.e., a data-driven technique---for such arrays. The novelty of this paper is twofold: we first revisit in situ calibration of mobile cross-sensitive sensors as an informed factorization of a partially observed non-negative matrix. A fast informed (semi-)NMF approach is then proposed and found to be well-suited for the considered problem.

In this paper the problem of single image indoor camera localization has been addressed. This is a difficult task, since no GPS is available and the training data being gathered for the indoor positioning system could be subject to many modifications such as occlusion, variation of illumination, or repetitive textures and patterns during the test, and these effects can easily fool any positioning system. In this paper, following the idea of self attention and the transformer networks, we customized the feature extraction system and the output extraction block of a recently used transformer in the image recognition task, so that to achieve the camera 3D position and 4D quaternion information. Moreover, an engineering implementation trick was employed, and the results were evaluated on the 7scenes dataset, and compared to the other state-of-the-art methods. The output results show a consistent outperformance with rather a simpler, and faster configuration.

This paper proposes an improved generalized knowledge distillation framework with multiple dissimilar teacher networks, each of which is specialized for a specific domain, to make a deployable student network more robust to challenging acoustic environments. In this paper, we first address a method to partition the training data for constructing ensembles of the teachers from unsupervised neural clustering with features based on context-dependent phonemes representing each acoustic domain. Second, we illustrate how a single student network designed from partitioned data is effectively trained with multiple specialized teachers. During the training step, the weights of the student network are updated using a composite two-part cross entropy loss obtained from a pair consisting of a specialized teacher corresponding to input speech and a generalized teacher trained with a balanced data set. Unlike system combination methods, we aim to incorporate the benefits from multiple models into a single student network via knowledge distillation that does not increase any computational costs during the decoding time. The improvement of the proposed technique is shown on acoustically diverse signals contaminated by challenging practical noises.

Information steganography is a family of techniques that hide secret messages into a carrier; thus, the messages can only be extracted by receivers with a correct key Although many approaches have been proposed to achieve this purpose, historically, it is a difficult problem to conceal a large amount of information without occasioning human perceptible changes. In this paper, we explore the room introduced by the low-rank property of natural signals (i.e., images, audios), and propose a training-free model for efficient information steganography, which provides a capacity of hiding full-size images into carriers of the same spatial resolution. The key of our method is to randomly shuffle the secrets and carry out a simple reduction summation with the carrier. On the other hand, the secret images can be reconstructed by solving a convex optimization problem similar to the ordinary tensor decomposition. In the experimental analysis, we carry out two tasks: concealing a full-RGB-color image into a gray-scale image; concealing images into music signals. The results confirm the ability of our model to handle massive secret payloads. The code of our paper is provided in https://github.com/minogame/icassp-SIC.

The paper presents an original solution to the online handwritten document processing in a free form, which is aimed at separating multi-class handwritten documents into texts, tables, formulas, drawings, etc. Stroke classification is an important step in automatic document layout analysis (DLA) in handwritten document recognition systems. Major DLA challenges arise due to a wide diversity of handwritten content, various writing styles, a lack of contextual knowledge, and the complicated structure of freeform handwritten documents. In this paper, we propose the hierarchical recurrent neural network (RNN) architecture to address the hierarchical structure inherent to the handwritten document. The novelty of feature aggregation pooling technique for transferring data between hierarchical levels allows achieving higher computational efficiency for using the suggested approach in on-device mobile computing. The presented approach gives an access to new state-of-the-art results in the task of multi-class classification with an accuracy of 97.25% on the IAMonDo dataset. This result can serve as the basis for efficient mobile applications for freeform handwriting document recognition.

Pruning is one of the major methods to compress deep neural networks. In this paper, we propose an Ising energy model within an optimization framework for pruning convolutional kernels and hidden units. This model is designed to reduce redundancy between weight kernels and detect inactive kernels/hidden units. Our experiments using ResNets, AlexNet, and SqueezeNet on CIFAR-10 and CIFAR-100 datasets show that the proposed method on average can achieve a pruning rate of more than $50\%$ of the trainable parameters with approximately $, This paper is accepted for presentation at IEEE International Conference on Acoustics, Speech and Signal Processing (IEEE ICASSP), 2021

Deep learning-based algorithms provide an efficient and reliable diagnosis for medical imaging. This paper proposes COVID-19 diagnosis based on analysis of Computerised tomography (CT) chest scans. In recent years, deep learning-based analysis of CT chest scans has demonstrated competitive sensitivity for pneumonia prognosis. This paper presents our submission for the 2021 ICASSP Signal Processing Grand Challenge (SPGC). We exploit a 3D Network-based transfer learning approach to classify volumetric CT scans with a novel pre-processing method to render the volume with salient features. This work uses the pre-trained 3D ResNet50 as the backbone network. The 3D network is trained on a dataset consisting of 3 classes: Community Acquired Pneumonia (CAP), COVID-19 and Normal patient. The final testing results have shown an overall accuracy of 85.56% with the COVID-19 sensitivity attaining 82.86%.

Lung cancer has been the leading death cause in modern society. Early detection of pulmonary nodules can significantly improve the survival rate of lung cancer. In this paper, we propose a novel pulmonary nodule detection framework and a novel 3D dense attention module (DAM) which can efficiently exploit the abundant 3D spatial features. The attention module, which integrates the improved dense block and the conv attention block, focuses on three dimensions, plane attention, depth attention, and channel attention. And the whole framework consists of two phases: Nodule Candidate Generation (NCG) and False Positive Reduction (FPR). In NCG phase, we construct a detection network based on DAM. Due to the wide distribution of the nodule diameters, we propose a 3D Feature Pyramid Network (3DFPN) to better handle the scale-varying problem. In FPR phase, we design a 3D DCNN to erase the false positives. Sliding-window based data augment methods are adopted to deal with the unbalance problem of the data. Comprehensive experiments show that our scheme outperforms the existing methods.

Collaborative filtering-based recommender systems are known to suffer from the item cold-start problem. Most recent attempts to mitigate this problem presented parametric approaches, such as deep content based models. In this paper, we show that a straightforward application of parametric models may lead to discrepancies between the cold and warm items’ distributions in the CF space. As a remedy, we propose to combine parametric with non-parametric estimation for robust cold item placement. Extensive evaluation indicates that our method is competitive with other baselines, while producing cold items placement that better resembles the distribution of warm items in the collaborative filtering space.

Knowledge distillation (KD) has enabled remarkable progress in model compression and knowledge transfer. However, KD requires a large volume of original data or their representation statistics that are not usually available in practice. Data-free KD has recently been proposed to resolve this problem, wherein teacher and student models are fed by a synthetic sample generator trained from the teacher. Nonetheless, existing data-free KD methods rely on fine-tuning of weights to balance multiple losses, and ignore the diversity of generated samples, resulting in limited accuracy and robustness. To overcome this challenge, we propose robustness and diversity seeking data-free KD (RDSKD) in this paper. The generator loss function is crafted to produce samples with high authenticity, class diversity, and inter-sample diversity. Without real data, the objectives of seeking high sample authenticity and class diversity often conflict with each other, causing frequent loss fluctuations. We mitigate this by exponentially penalizing loss increments. With MNIST, CIFAR-10, and SVHN datasets, our experiments show that RDSKD achieves higher accuracy with more robustness over different hyperparameter settings, compared to other data-free KD methods such as DAFL, MSKD, ZSKD, and DeepInversion., Accepted in IEEE ICASSP 2021

It is challenging to remove rain steaks from a single rainy image because the rain steaks are spatially varying in the rainy image. On top of a new insight in single image de-raining, a nonlocal de-raining algorithm is proposed in this paper to remove the rain streaks from the rainy image. The rainy image is not decomposed into different layers by the proposed algorithm. Experimental results validate the proposed algorithm.

Physiological studies have shown that differences in facial activities between depressed patients and normal individuals are manifested in different local facial regions and the durations of these activities are not the same. But most previous works extract features from the entire facial region at a fixed time scale to predict the individual depression level. Thus, they are inadequate in capturing dynamic facial changes. For these reasons, we propose a multi-scale and multi-region fa-cial dynamic representation method to improve the prediction performance. In particular, we firstly use multiple time scales to divide the original long-term video into segments containing different facial regions. Secondly, the segment-level feature is extracted by 3D convolution neural network to characterize the facial activities with different durations in different facial regions. Thirdly, this paper adopts eigen evolution pooling and gradient boosting decision tree to aggregate these segment-level features and select discriminative elements to generate the video-level feature. Finally, the depression level is predicted using support vector regression. Experiments are conducted on AVEC2013 and AVEC2014. The results demonstrate that our method achieves better performance than the previous works.

Research on automated diagnosis of Coronavirus Disease 2019 (COVID-19) has increased in recent months. SPGC COVID19 aims at classifying the grouped images of the same patient into COVID, Community Acquired Pneumonia(CAP) or normal. In this paper, we propose a novel ensemble learning framework to solve this problem. Moreover, adaptive boosting and dataset clustering algorithms are introduced to improve the classification performance. In our experiments, we demonstrate that our framework is superior to existing networks in terms of both accuracy and sensitivity.

The image source identification task is mainly addressed by exploiting the unique traces of the sensor pattern noise, that ensure a negligible false alarm rate when comparing patterns extracted from different devices, even of the same brand or model. However, most recent smartphones are equipped with proprietary in-camera processing that can possibly expose unexpected correlated patterns within images belonging to different sensors.In this paper, we first highlight that wrong source attribution can happen on smartphones belonging to the same brand when images are acquired both in default and in bokeh mode. While the bokeh mode is proved to introduce a correlated pattern due to the specific in-camera post-processing, we also show that natural images also expose such issue, even when a reference from flat images is available. Furthermore, different camera models expose different correlation patterns since they are reasonably related to developers’ choices. Then, we propose a general strategy that allows the forensic practitioner to determine whether a questioned device may suffer from these correlated patterns, thus avoiding the risk of false image attribution.

Current light field (LF) occlusion removal approaches usually select only a part of sub-aperture images (SAIs) or simply stack all SAIs to reconstruct the center view, which destroys the spatial layout of SAIs. In this paper, we present a simple yet effective LF occlusion removal method name Mask4D, which is a 4D convolution-based encoder-decoder network. We propose to keep the spatial layout of SAIs and construct all SAIs as a 5D input tensor to fully exploit the spatial connection information between SAIs. In particular, except for center view reconstruction, we jointly predict the occlusion mask to disentangle the occlusion mask from the occluded content. Extensive evaluations demonstrate that our Mask4D surpasses the state-of-the-art approaches across different datasets. Moreover, visualizations show that Mask4D predicts the occlusion mask precisely and the reconstructed center view looks more realistic than other approaches. Our code will be publicly available.

Hospital workers are known to work long hours in a highly stressful environment. The COVID-19 pandemic has increased this burden multi-fold. Pre-COVID statistics already showed that one in every three nurses reported burnout, thus affecting patient satisfaction and the quality of their provided service. Real-time monitoring of burnout, and other underlying factors, such as stress, could provide feedback not only to the clinical staff, but also to hospital administrators, thus allowing for supportive measures to be taken early. In this paper, we present a context-aware speech-based system for stress detection. We consider data from 144 hospital workers who were monitored during their daily shifts over a 10-week period; subjective stress readings were collected daily. Wearable devices measured speech features and physiological readings, such as heart rate. Environment sensors, in turn, were used to track staff movement within the hospital. Here, we show the importance of context-awareness for stress level detection based on a bidirectional LSTM deep neural network. In particular, we show the importance of hospital location and circadian rhythm based contextual cues for stress prediction. Overall, we show improvements as high as 14% in F1 scores once context is incorporated, relative to using the speech features alone.

Restoring Arabic syntactic diacritics based on Long Short-Term Memory (LSTM) networks leads to state-of-the-art performance. These LSTM networks are commonly augmented with Maximum Entropy (MaxEnt) sparse direct connections between the input and the output layers of the tagger. One way to improve such tagger performance is to use an ensemble of taggers. However, an ensemble of taggers may require huge computational and memory resources. In this paper, we implement a knowledge distillation technique where an ensemble of teachers/taggers is used to train a single student tagger. On the other hand, Arabic is a morphologically rich language and has a high Out-Of-Vocabulary (OOV) rate. In addition to word embeddings, we propose to use character embeddings encoded using LSTMs for each word to overcome this problem. On the Arabic tree bank task, our hybrid LSTM/MaxEnt tagger achieves 1.0% absolute WER improvement over a strong baseline using the proposed two techniques.

Object counting has become an important task in computer vision for its practical applications in surveillance system. Previous methods for object counting have achieved promising results in accuracy, but few researchers focus on the real-time performance of counting methods. In this paper, we propose an efficient and accurate light-weight network for object counting, called Partial Feature Aggregation Network (PFANet). In this novel method, a Partial Feature Aggregation (PFA) structure is designed to accelerate networks and improve the utilization of multi-scale features. Moreover, PFANet uses the dilated convolution to enlarge the receptive-filed of network. Experiments on two datasets indicate our network exceeds the existing real-time counting networks in both accuracy and efficiency.

With great research advances on Brain-Computer-Interface (BCI) systems, Electroencephalography (EEG) based driver fatigue state classification models have shown its effectiveness. However, EEG signals contain large differences between individuals, making it hard to build a unified model among individuals. In this paper, we propose a subject- independent EEG-based driver fatigue state (i.e., awake, tired, and drowsy) classification model that mitigates a performance gap between subjects. To this end, we exploit an adversarial training strategy to make our classification model misclassify the subject labels. Besides, we propose an Intersubject Feature Distance Minimization (IFDM) method that minimizes the Wasserstein distance between two different subject groups of the same class to reduce the individual performance discrepancy. Our method is also designed to enable training even if the subject labels are not sufficiently included in the EEG dataset. To demonstrate the ability of the proposed method, we conduct a drowsiness classification task on a publicly available SEED-VIG dataset. The experimental results show our model achieves the highest accuracy and the lowest individual performance variability.

Automatic radar based classification of automotive targets, such as pedestrians and cyclist, poses several challenges due to low inter-class variations among different classes and large intra-class variations. Further, different targets required to track in typical automotive scenario can have completely varying dynamics which gets challenging for tracker using conventional state vectors. Compared to state-of-the-art using independent classification and tracking, in this paper, we propose an integrated tracker and classifier leading to a novel Bayesian framework. The tracker’s state vector in the proposed framework not only includes the localization parameters of the targets but is also augmented with the targets’s feature embedding vector. In consequence, the tracker’s performance is optimized due to a better separability of the targets. Furthermore, the classifier’s performance is enhanced due to Bayesian formulation utilizing the temporal smoothing of classifier’s embedding vector.

Aerial image classification is challenging for current deep learning models due to the varied geo-spatial object scales and the complicated scene spatial arrangement. Thus, it is necessary to stress the key local feature response from a variety of scales so as to represent discriminative convolutional features. In this paper, we propose a deep multi-scale multiple instance learning (DMSMIL) framework to tackle the above challenges. Firstly, we develop a differential multi-scale dilated convolution feature extractor to exploit the different patterns from different scales. Then, the deep features of each scale are fed into a multiple instance learning module to generate a bag-level probability prediction. Lastly, probability predictions from all the MIL branches are fused to generate the final semantic prediction. Extensive experiments on three widely-utilized aerial scene classification benchmarks demonstrate that our proposed DMSMIL outperforms the state-of-the-art approaches by a large margin.

Linear inverse problems appear in many applications, where different algorithms are typically employed to solve each inverse problem. Nowadays, the rapid development of deep learning (DL) provides a fresh perspective for solving the linear inverse problem: a number of well-designed network architectures results in state-of-the-art performance in many applications. In this overview paper, we present the combination of the DL and the Plug-and-Play priors (PPP) framework, showcasing how it allows solving various inverse problems by leveraging the impressive capabilities of existing DL based denoising algorithms. Open challenges and potential future directions along this line of research are also discussed.

Anatomical landmark localization has been a key challenge for medical image analysis. Existing researches mostly adopt CNN as the main architecture for landmark localization while they are not applicable to process image modalities with periodic structure. In this paper, we propose a novel two-stage frame-level detection and heatmap regression model for accurate landmark localization in m-mode echocardiography, which promotes better integration between global context information and local appearance. Specifically, a periodic frame detection module with LSTM is designed to model periodic context and detect frames of systole and diastole from original echocardiography. Next, a CNN based heatmap regression model is introduced to predict landmark localization in each systolic or diastolic local region. Experiment results show that the proposed model achieves average distance error of 9.31, which is at a reduction by 24% comparing to baseline models.

This paper focuses on the problem of temporal dependency modeling in the CNN-based models for audio classification tasks. To capture audio temporal dependencies using CNNs, we take a different approach from the purely architecture-induced method and explicitly encode temporal dependencies into the CNN-based audio classifiers. More specifically, in addition to the classification objective, we require the CNN model to solve an auxiliary task of predicting the future features, which is formulated by leveraging the Contrastive Predictive Coding (CPC) loss. Furthermore, a novel hierarchical CPC (HCPC) model is proposed for capturing multi-level temporal dependencies at the same time. The proposed model is evaluated on a wide range of non-speech audio signals, including musical and in-the-wild environmental audio signals. We show that the proposed approach improves the backbone CNNs consistently on all tested benchmark datasets and outperforms a DenseNet model trained from scratch.

Identifying uterine contractions with the aid of machine learning methods is necessary vis-á-vis their use in combination with fetal heart rates and other clinical data for the assessment of a fetus wellbeing. In this paper, we study contraction identification by processing noisy signals due to uterine activities. We propose a complete four-step method where we address the imbalanced classification problem with an ensemble Gaussian process classifier, where the Gaussian process latent variable model is used as a decision-maker. The results of both simulation and real data show promising performance compared to existing methods.

This paper is devoted to the recently proposed mixing model with constant separating vector (CSV) for Blind Source Extraction of moving sources using the FastDIVA algorithm, which is an extension of the famous FastICA and FastIVA for static mixtures. The benefits due to the CSV model and FastDIVA are demonstrated in three new applications. First, the extraction of a moving speaker in a noisy reverberant environment using a dense array of 48 MEMS microphones is considered. Second, a case study on the blind extraction of moving brain activity from visually evoked potentials in electroencephalogram is reported. Third, a simulation of block-by-block online extraction of a moving source is demonstrated. In these examples, the CSV and FastDIVA show their new potential and good performance in handling the blind moving source extraction problem.

Nowadays, web data is often used to train a dialogue system. However, noises in web data can disturb the training process, as well as can impact the performance. Consequently, dialogue models tend to be brittle when receiving noisy inputs during the inference. This paper proposes a novel framework, Multi-Path Training (MPT), for training a robust dialogue response generation system. MPT improves the robustness to the noisy training data and the noisy inference queries using three paths. Experimental results show MPT can outperform baselines using the same backbone model, and also prove MPT can improve the robustness to the noise in both the training and inference stage.

Sub-Nyquist sampling is used in modern high-resolution pulse-Doppler radar systems to reduce system resources and improve resolution. Xampling with Doppler focusing is utilized to implement these sub-Nyquist radar systems. Signal recovery involves iterative optimization requiring large computational time that may be prohibitive in real applications. In this paper, we propose Deep Unfolded Radar Sensing (DURAS), a model-based deep learning architecture to address this problem. We utilize the recently introduced complex LISTA (C-LISTA) with recurrent neural network units and complex soft-thresholding to handle the complex-valued measurement signals. We propose a partial Doppler focusing (PDF) framework with ensembling of multiple PDF measurement vectors via a convolutional neural network (CNN). This CNN followed by a complex cardioid activation function is added to the front end of the C-LISTA architecture. Thus, DURAS is a hybrid architecture of partial Doppler focusing, CNN, and C-LISTA that provides considerably improved performance compared to existing methods on target detection in radar systems.

Fact verification is a challenging task which requires to retrieve relevant sentences from plain texts and then take these sentences as evidences to verify given claims. Conventional methods treat sentence selection and claim verification as separate subtasks in a pipeline. Claim verification models usually analyzes the inference relationship between each retrieved sentence and the claim, and then aggregates the claim-sentence representations by graph-based reasoning methods, such as graph attention networks (GAT). In this paper, we propose a graph attention and interaction network (GAIN) for claim verification. In addition to GAT, this model includes a graph interaction network (GIN), which considers the comparative relationships among all claim-sentence representations. More importantly, a multi-task learning strategy, which combines the objectives of both sentence selection and claim verification, is designed to train the GAIN model in order to utilize the supervision information of both subtasks. Experimental results on the FEVER dataset show that the GAIN model with multi-task learning achieves a FEVER score of 73.04%, which outperforms other published models.

With the recent advancement of the Brain-Computer Interface (BCI), Electroencephalogram (EEG) analytics gain a lot of research attention from various domains. Understanding the vulnerabilities of EEG analytics is important for safely applying this emerging technology in our daily life. Recent studies show that EEG analytics are vulnerable to adversarial attacks when adding small perturbations on the EEG data. However, fewer research efforts have been devoted to the robustness of EEG analytics under sparse perturbations that attack only small portions of the data. In this paper, we conduct the first in-depth study on the robustness of EEG analytics under sparse perturbations and propose the first Sparse Adversarial eeG Attack, SAGA, to identify weakness of EEG analytics. Specifically, by viewing EEG data as time series collected from several channels, we design an adaptive mask to uniformly represent diverse sparsity in adversarial attacks. We further introduce a PGD-based iterative solver to automatically select the time steps and channels under the given sparsity constraints and effectively identify the adversarial examples on EEG data. Extensive experiments show that SAGA can effectively generate sparse perturbations and introduces a 77.02% accuracy drop on average by only perturbing 5% channels and time steps.

In these days, more and more deep learning models are applied to various tasks in an industrial area. Because sensors are changed rapidly, if we want to use the model then we need to collect new sensor data and train with them again. Moreover, sometimes we need to tune the model for the task and adjust the hyperparameters. In this paper, we propose a signal translation networks, Sig2Sig, that converts from the new sensor signals to old ones in order to reuse the past model, which was trained on plenty of old sensor signals. We only need a small paired-dataset for training Sig2Sig and then translated signals from a new sensor can be classified by the past classification model almost without performance degradation. To do such a robust translation, our model is based on a modified u-net with squeeze and excitation networks, instance normalization, and with attention parts for robust outputs. We also use various loss functions which include uncertainty loss for ignoring noise parts and focusing on important parts of signal images. Our results show that not only translated signal is realistic and similar to target data but also the past classification model can be reused on the new sensor domain with almost similar performance as the old sensor.

Automatic pavement disease recognition is vital for pavement maintenance and management. In this paper, we present an end-to-end deep learning approach named Weakly Super-vised Patch Label Inference Network with Image Pyramid (WSPLIN-IP) for recognizing various types of pavement diseases that are not just limited to the specific ones, such as crack and pothole. WSPLIN-IP first divides the pavement image into patches with an image pyramid for fully exploiting the resolution and scale information. Then, a Patch Label Inference Network (PLIN) is employed for inferring the labels of these patches constrained with a patch label sparsity loss. Finally, the patch labels are fed into a Comprehensive Decision Network (CDN) for disease recognition. Since only the image label is available during whole training, the training of PLIN is conducted in a weakly supervised way under the guidance of CDN and the trained PLIN can provide the interpretable intermediate information. We evaluate our method on a large-scale Bituminous Pavement Disease Dataset named CQU-BPDD whose samples are acquired in the real world. Extensive results demonstrate the superiority of our method over baselines.

Freezing of Gait (FoG) is a paroxysmal and devitalizing symptom associated with Parkinson’s disease (PD). Episodes of FoG impedes gait and augments fall propensity, often leading to serious fall-injury. In this paper, we present a method for online detection of FoG using a wearable motion sensor. The novelty lies in utilizing the Empirical Wavelet Transform for signal denoising and incorporating the two new features to ameliorate the accuracy of the algorithm. Fundamentally, we have focused on a patient-independent model and leveraged a single ankle sensor which makes it a more feasible approach in terms of usability. Our model is evaluated on Daphnet dataset and achieved the average Sensitivity of.95 and Specificity of.70 with only a single sensor, demonstrating its immense potential.

Slot filling and intent detection have become a significant theme in the field of natural language understanding. Even though slot filling is intensively associated with intent detection, the characteristics of the information required for both tasks are different while most of those approaches may not fully aware of this problem. In addition, balancing the accuracy of two tasks effectively is an inevitable problem for the joint learning model. In this paper, a Continual Learning Interrelated Model (CLIM) is proposed to consider semantic information with different characteristics and balance the accuracy between intent detection and slot filling effectively. The experimental results show that CLIM achieves state-of-the-art performace on slot filling and intent detection on ATIS and Snips., Comment: Accepted to ICASSP 2021

To meet the imperative demand for monitoring the quality of Ultra High-Definition (UHD) content in multimedia industries, we propose an efficient no-reference (NR) image quality assessment (IQA) metric to distinguish original and pseudo 4K contents and measure the quality of their quality in this paper. First, we establish a database including more than 3000 4K images composed of natural 4K images together with upscaled versions interpolated from 1080p and 720p images by fourteen algorithms. To improve computing efficiency, our model segments the input image and selects three representative patches by local variances. Then, we extract the histogram features and cut-off frequency features in the frequency domain as well as the natural scenes statistic (NSS) based features from the representative patches. Finally, we employ support vector regressor (SVR) to aggregate these extracted features as an overall quality metric to predict the quality score of the target image. Extensive experimental comparisons using seven common evaluation indicators demonstrate that the proposed model outperforms the competitive NR IQA methods and has a great ability to distinguish true and pseudo 4K images.

Recent advances in solving imaging inverse problems have witnessed the combination of deep learning models with classical image models for better signal representation. One such approach, DeepRED, combines the deep image prior (DIP) with the regularization by denoising (RED) framework to boost the performance of image deblurring and super resolution tasks. In this paper, we formulate DeepRED as a consensus equilibrium problem and set up a fixed-point algorithm for solving the equilibrium equations. We also derive sufficient conditions that the DIP generative prior should satisfy to ensure that the corresponding fixed-point operator is non-expansive. We then demonstrate that the fixed-point algorithm that solves the CE equations results in improved image reconstruction quality in a deblurring setting compared to state-of-the-art methods.

Research on designing local feature descriptors has gradually shifted to deep learning. Different from other computer vision tasks, the biggest challenge for local descriptor learning lies with the formulation of loss functions. Existing methods solve the problem by leveraging Siamese loss or triplet loss and improve the performance of the learned descriptors by a significant margin. However, the widely used Siamese loss and triplet loss cannot fully utilize the context information. In this paper, we propose a novel loss function to introduce more context information to facilitate training. After incorporating the proposed loss function into training, our learned descriptor demonstrates state-of-the-art performance in patch verification, image matching and patch retrieval benchmarks. The pretrained model will be publicly available at https://github.com/clelouch/CANet.

Direct-path relative transfer function (DP-RTF) refers to the ratio between the direct-path acoustic transfer functions of two channels. Though DP-RTF fully encodes the sound directional cues and serves as a reliable localization feature, it is often erroneously estimated in the presence of noise and reverberation. This paper proposes a supervised DP-RTF learning method with deep neural networks for robust binaural sound source localization. To exploit the complementarity of single-channel spectrogram and dual-channel difference information, we first recover the direct-path magnitude spectrogram from the contaminated one using a monaural enhancement network, and then predict the DP-RTF from the dual-channel (enhanced-) intensity and phase cues using a binaural enhancement network. In addition, a weighted-matching softmax training loss is designed to promote the predicted DP-RTFs to be concentrated for the same direction and separated for different directions. Finally, the direction of arrival (DOA) of source is estimated by matching the predicted DP-RTF with the ground truths of candidate directions. Experimental results show the superiority of our method for DOA estimation in the environments with various levels of noise and reverberation.

Change detection plays a vital role in monitoring and analyzing temporal changes in Earth observation tasks. This paper proposes a novel adaptive multi-scale and multi-level features fusion network for change detection in very-high-resolution bi-temporal remote sensing images. The proposed approach has three advantages. Firstly, it excels in abstracting high-level representations empowered by a highly effective feature extraction module. Secondly, an elaborate feature fusion module incorporated with the channel and spatial attention mechanism is proposed to provide efficient fusion strategies for multi-scale and multi-level features from bi-temporal images and multiple convolutional layers. Finally, a novel perceptual auxiliary component is designed to capture the perceptual loss of the global perceptual and structural differences and address the optimization problem caused by only using per-pixel loss function in change detection. Comprehensive experiments on two benchmark datasets confirm that our proposed framework outperforms state-of-the-art algorithms in both quantitative assessment and visual interpretation.

In practical application, the existing gender and age recognition algorithms can’t meet the requirements of both smallsized model and high accuracy simultaneously. Moreover, most models based on CNNs have a even larger size of more than 200M. In this paper a multi-layer multi-channel attentive network based on the idea of divide-and-conquer is proposed. This method uses multi-layer processes in each channel to extract features of different layers and improves accuracy by layer refinement. We use some dynamic parameters to fine- tune each layer to make the model fit better. Each layer uses the same classifier to reduce parameters to make the model smaller. And we import attention mechanisms to increase the ability of the network to use the features. Experiments show that the accuracy of this method is better than several mainstream networks and the size of the model is less than 0.5M, which can be used in mobile terminals well.

Packet loss may affect a wide range of applications that use voice over IP (VoIP), e.g. video conferencing. In this paper, we investigate a time-domain convolutional recurrent network (CRN) for online packet loss concealment. The CRN comprises a convolutional encoder-decoder structure and long short-term memory (LSTM) layers, which have been shown to be suitable for real-time speech enhancement applications. Moreover, we propose lookahead and masked training to further improve the performance of the CRN framework. Experimental results show that the proposed system outperforms a baseline system using only LSTM layers in terms of two objective metrics – perceptual evaluation of speech quality (PESQ) and short-term objective intelligibility (STOI); it also reduces the word error rate (WER) more than the baseline when used as a frontend for speech recognition. The advantage of the proposed system is also verified in a subjective evaluation by the mean opinion score (MOS).

Using only one deep model for cross scene video foreground segmentation is still very challenging because existing methods are scene-dependent, which restricts the consistent segmentation. In this paper, we propose a cross scene video foreground segmentation framework to extend the generalization capability of those supervised model depending on scene-specific training. The proposed framework flexibly utilizes three well-trained supervised models as guidance to yield a coarse segmentation mask. The co-occurrence probability-based unsupervised background subtraction model is introduced to achieve scene adaptation in the plug and play style without any fine-tuning and labels. Experimental results on LIMU and CDNet2014 datasets validate our framework outperforms the state-of-the-art supervised/unsupervised approaches that participate in the comparison. Experiments also show the training efficiency-related improvements – when introducing the guidance models, the demand for quantity and quality of training samples to train the unsupervised model is reduced. Codes https://github.com/MeteoorLiu/Venus/tree/MeteoorLiu-SUMC

Domain mapping aims to estimate a sophisticated mapping between source and target domains. Finding the specialized attribute in latent representation plays a key role to attain a desirable performance. However, the entangled features usually contain the mixed attribute which can not be easily decomposed in an unsupervised manner. To handle the mixed features for better generation, this paper presents an attribute decomposition based on the sequence data and carries out the flow-based image domain mapping. The latent variables, characterized by flow model, are decomposed into the attribute-relevant and attribute-irrelevant components. The decomposition is guided by multiple objectives including structural-perceptual loss, cycle consistency loss, sequential random-pair reconstruction loss and sequential classification loss where the paired training data for domain mapping are not required. Importantly, the sequential random-pair reconstruction loss is formulated by means of exchanging the attribute-relevant components within a sequence of images. As a result, the source images with the attributes of reference images can be smoothly transferred to the corresponding target images. Experiments on talking face synthesis show the merit of attribute decomposition in domain mapping.