Search

Missing data imputation plays an important role in the data cleansing process. Clustering algorithms have been widely used for missing data imputation, yet, there is little research done on the use of clustering ensemble for missing data imputation, which aggregates multiple clustering results. This paper proposes a novel collaborative clustering-based imputation method, called COLI, which uses the imputation quality as a key criterion for the exchange of information between different clustering results. To the best of our knowledge, this is the first study on the impact of collaborative clustering on imputation performance. The main contributions of this paper are three-fold. A novel missing value imputation based on collaborative clustering is proposed, three amputation strategies are used to induce missingness on various complete and publicly available datasets with different mechanisms, distributions, and ratios, which allows evaluating the imputation quality of the proposed method in estimating missing values of various numerical datasets with different missingness mechanisms, distributions, and ratios. The proposed method is compared to several state-of-the-art imputation methods and attained results demonstrate that the proposed method is an effective method for handling missing data.

Automatic fingerprint identification is a key technology of biological attribute measurement. Due to its sensitivity to fake fingerprint attack, its security has been widely concerned. The fingermark is an important evidence for forensic documents identification. The existence of fake fingermark seriously threatens the fairness and legitimacy in the process of forensic identification. In this paper, in view of the security risks existing in the fingermark identification of forensic documents, a database named JLW-FM-DB is introduced for detecting genuine and fake fingermarks, which consists of two sub databases, signed and unsigned, each of which covers common fake fingermark materials. Based on the database, this paper proposes a method of fingermark anti-counterfeiting based on convolution neural network(CNN). A Patch-Label training strategy is proposed, which uses the unified label as the supervision signal for the class heatmap output by the last convolution layer. This strategy realizes stronger local supervision ability to input fingermark image and enhances local coding ability of CNN feature extraction. The experiments show that our methods are suitable for the detection of genuine and fake fingermarks, achieving 97.999% average accuracy in the signed fingermark database. The effectiveness of Patch-Label training strategy in fingermark anti-counterfeiting is also proved. Moreover, a model fusion of different models can further improve the detection ability of genuine and fake fingermarks, reaching to 98.496% average accuracy.

Recently, deep learning models have achieved great success in computer vision applications, relying on large-scale class-balanced datasets. However, imbalanced class distributions still limit the wide applicability of these models due to degradation in performance. To solve this problem, in this paper, we concentrate on the study of cross entropy which mostly ignores output scores on incorrect classes. This work discovers that neutralizing predicted probabilities on incorrect classes improves the prediction accuracy for imbalanced image classification. This paper proposes a simple but effective loss named complement cross entropy based on this finding. The proposed loss makes the ground truth class overwhelm the other classes in terms of softmax probability, by neutralizing probabilities of incorrect classes, without additional training procedures. Along with it, this loss facilitates the models to learn key information especially from samples on minority classes. It ensures more accurate and robust classification results on imbalanced distributions. Extensive experiments on imbalanced datasets demonstrate the effectiveness of the proposed method., 8 pages, Accepted to Pattern Recognition Letters (PRL), August 2021

This paper presents a new model for the task of scene text visual question answering, in which questions about a given image can only be answered by reading and understanding scene text that is present in it. The proposed model is based on an attention mechanism that attends to multi-modal features conditioned to the question, allowing it to reason jointly about the textual and visual modalities in the scene. The output weights of this attention module over the grid of multi-modal spatial features are interpreted as the probability that a certain spatial location of the image contains the answer text the to the given question. Our experiments demonstrate competitive performance in two standard datasets. Furthermore, this paper provides a novel analysis of the ST-VQA dataset based on a human performance study., Comment: This paper is under consideration at Pattern Recognition Letters

In this paper, we tackle the task of improving biometric verification in the context of Human-Robot Interaction (HRI). A robot that wants to identify a specific person to provide a service can do so by either image verification or, if light conditions are not favourable, through voice verification. In our approach, we will take advantage of the possibility a robot has of recovering further data until it is sure of the identity of the person. The key contribution is that we select from both image and audio signals the parts that are of higher confidence. For images we use a system that looks at the face of each person and selects frames in which the confidence is high while keeping those frames separate in time to avoid using very similar facial appearance. For audio our approach tries to find the parts of the signal that contain a person talking, avoiding those in which noise is present by segmenting the signal. Once the parts of interest are found, each input is described with an independent deep learning architecture that obtains a descriptor for each kind of input (face/voice). We also present in this paper fusion methods that improve performance by combining the features from both face and voice, results to validate this are shown for each independent input and for the fusion methods.

Images with visual and scene text content are ubiquitous in everyday life. However, current image interpretation systems are mostly limited to using only the visual features, neglecting to leverage the scene text content. In this paper, we propose to jointly use scene text and visual channels for robust semantic interpretation of images. We do not only extract and encode visual and scene text cues, but also model their interplay to generate a contextual joint embedding with richer semantics. The contextual embedding thus generated is applied to retrieval and classification tasks on multimedia images, with scene text content, to demonstrate its effectiveness. In the retrieval framework, we augment our learned text-visual semantic representation with scene text cues, to mitigate vocabulary misses that may have occurred during the semantic embedding. To deal with irrelevant or erroneous recognition of scene text, we also apply query-based attention to our text channel. We show how the multi-channel approach, involving visual semantics and scene text, improves upon state of the art., Comment: The paper is under consideration at Pattern Recognition Letters

The paper presents Imbalance-XGBoost, a Python package that combines the powerful XGBoost software with weighted and focal losses to tackle binary label-imbalanced classification tasks. Though a small-scale program in terms of size, the package is, to the best of the authors' knowledge, the first of its kind which provides an integrated implementation for the two losses on XGBoost and brings a general-purpose extension on XGBoost for label-imbalanced scenarios. In this paper, the design and usage of the package are described with exemplar code listings, and its convenience to be integrated into Python-driven Machine Learning projects is illustrated. Furthermore, as the first- and second-order derivatives of the loss functions are essential for the implementations, the algebraic derivation is discussed and it can be deemed as a separate algorithmic contribution. The performances of the algorithms implemented in the package are empirically evaluated on Parkinson's disease classification data set, and multiple state-of-the-art performances have been observed. Given the scalable nature of XGBoost, the package has great potentials to be applied to real-life binary classification tasks, which are usually of large-scale and label-imbalanced., Updated version, published in Pattern Recognition Letters 136(2020):190-197

In recent days, the incidence of Diabetic Retinopathy (DR)has become high, affecting the eyes because of drastic increase in the glucose level in blood. Globally, almost half of the people under the age of 70 gets severely affected by diabetes. In the absence of earlier recognition and proper medication, the DR patients tend to lose their vision. When the warning signs are tracked down, the severity level of the disease has to be validated so to take decisions regarding appropriate treatment further. The current research paper focuses on the concept of classification of DR fundus images on the basis of severity level using a deep learning model. This paper proposes a deep learning-based automated detection and classification model for fundus DR images. The proposed method involves various processes namely preprocessing, segmentation and classification. The methods begins with preprocessing stage in which unnecessary noise that exists in the edges is removed. Next, histogram-based segmentation takes place to extract the useful regions from the image. Then, Synergic Deep Learning (SDL) model was applied to classify the DR fundus images to various severity levels. The justification for the presented SDL model was carried out on Messidor DR dataset. The experimentation results indicated that the presented SDL model offers better classification over the existing models.

Component tree is a full image representation which encodes all connected components from upper (resp. lower) level sets of a given image through the inclusion relation. Information from this representation can be used in many image processing and computational vision applications, e.g. connected filtering, image segmentation, feature extraction, among others. In general, each node of a component tree represents a connected component of a level set and stores attributes which describes features of this connected component. This paper presents a review of a previously published method to compute attributes such as area, perimeter, and number of Euler by incrementally counting patterns while traversing nodes of a component tree. This method foundation is further detailed in this paper by presenting a novel theoretical background and algorithm correctness intuition. We also present a novel approach for this algorithm showing improvements for run-time execution and precision analysis.

We address the problem of spatio-temporal action localization in videos in this paper. Current state-of-the-art methods for this challenging task rely on an object detector to localize actors at frame-level firstly, and then link or track the detections across time. Most of these methods commonly pay more attention to leveraging the temporal context of videos for action detection while ignoring the importance of the object detector itself. In this paper, we prove the importance of the object detector in the pipeline of action localization, and propose a strong object detector for better action localization in videos, which is based on the single shot multibox detector (SSD) framework. Different from SSD, we introduce an anchor refine branch at the end of the backbone network to refine the input anchors, and add a batch normalization layer before concatenating the intermediate feature maps at frame-level and after stacking feature maps at clip-level. The proposed strong detector have two contributions: (1) reducing the phenomenon of missing target objects at frame-level; (2) generating deformable anchor cuboids for modeling temporal dynamic actions. Extensive experiments on UCF-Sports, J-HMDB and UCF-101 validate our claims, and we outperform the previous state-of-the-art methods by a large margin in terms of frame-mAP and video-mAP, especially at a higher overlap threshold.

Real-world data such as digital images, MRI scans and electroencephalography signals are naturally represented as matrices with structural information. Most existing classifiers aim to capture these structures by regularizing the regression matrix to be low-rank or sparse. Some other methodologies introduce factorization technique to explore nonlinear relationships of matrix data in kernel space. In this paper, we propose a multi-distance support matrix machine (MDSMM), which provides a principled way of solving matrix classification problems. The multi-distance is introduced to capture the correlation within matrix data, by means of intrinsic information in rows and columns of input data. A complex hyperplane is established upon these values to separate distinct classes. We further study the generalization bounds for i.i.d. processes and non i.i.d. process based on both SVM and SMM classifiers. For typical hypothesis classes where matrix norms are constrained, MDSMM achieves a faster learning rate than traditional classifiers. We also provide a more general approach for samples without prior knowledge. We demonstrate the merits of the proposed method by conducting exhaustive experiments on both simulation study and a number of real-word datasets., The paper is under consideration at Pattern Recognition Letters

The paper presents experiments with the home-made, low-cost prototype of a sensor-equipped pen for handwriting-based biometric authentication. The pen allows to capture the dynamics of user writing on normal paper, while producing a kind of password (passphrase) chosen in advance. The use of a word of any length instead of the user's signature makes the approach more robust to spoofing, since there is no repetitive pattern to steal. Moreover, if the template gets violated, this is much less harmful than signature catch. The entailed sensors are a pair of accelerometer and gyroscope and a pressure sensor. The aim is a natural yet precise interaction, that allows recognizing the user by the signals recorded while producing a specific word chosen during enrollment and possibly changed later. The pen can be exploited in a number of applications requiring user recognition, yet relieving from the need to learn complex procedures, and to undergo critical capture operations. The approach fuses the use of a kind of password, though not necessarily complex as those requested by traditional approaches, and biometric recognition. The novelty with respect to most proposals in literature is the combination of three elements at once: the matching of any handwritten text instead of user signature, the on-line capture of seven sensor signals to recognize handwriting dynamics (three from accelerometer, three from gyroscope and one from pressure sensor), and the use of normal paper instead of a digitizing tablet. Presented experiments test two different recognition techniques, implemented by two modules that can be alternatively plugged into the system. An SVM-based verification module entails to extract the most relevant features from writing dynamics, and to acquire a sufficient amount of enrolling data (30 samples per user) to train an SVM for each user. A pure Dynamic Time Warping (DTW) verification module does not require such training, and is tested using either a gallery with the same number of templates per user as those used for SVM training, or with a gallery containing a much lower number of templates per user (namely 5). Obtained results encourage further investigation of lightweight strategies for written password dynamics recognition.

Tensor train is a hierarchical tensor network structure that helps alleviate the curse of dimensionality by parameterizing large-scale multidimensional data via a set of network of low-rank tensors. Associated with such a construction is a notion of Tensor Train subspace and in this paper we propose a TT-PCA algorithm for estimating this structured subspace from the given data. By maintaining low rank tensor structure, TT-PCA is empirically more robust to noise as compared to PCA or Tucker-PCA. This is borne out numerically by testing the proposed approach on the Extended YaleFace Dataset B, MINIST Dataset, CIFAR-10 dataset. This paper shows that the TT-PCA methods achieve less storage requirements, and have computationally faster online implementation with improved classification performance.

In this paper, we introduce a new likelihood ratio imbalance degree (LRID) to measure the class-imbalance extent of multi-class data. Imbalance ratio (IR) is usually used to measure class-imbalance extent in imbalanced learning problems. However, IR cannot capture the detailed information in the class distribution of multi-class data, because it only utilises the information of the largest majority class and the smallest minority class. Imbalance degree (ID) has been proposed to solve the problem of IR for multi-class data. However, we note that improper use of distance metric in ID can have harmful effect on the results. In addition, ID assumes that data with more minority classes are more imbalanced than data with less minority classes, which is not always true in practice. Thus ID cannot provide reliable measurement when the assumption is violated. In this paper, we propose a new metric based on the likelihood-ratio test, LRID, to provide a more reliable measurement of class-imbalance extent for multi-class data. Experiments on both simulated and real data show that LRID is competitive with IR and ID, and can reduce the negative correlation with F1 scores by up to 0.55.

Since many important real-world classification problems involve learning from unbalanced data, the challenging class-imbalance problem has lately received considerable attention in the community. Most of the methodological contributions proposed in the literature carry out a set of experiments over a battery of specific datasets. In these cases, in order to be able to draw meaningful conclusions from the experiments, authors often measure the class-imbalance extent of each tested dataset using imbalance-ratio, i.e. dividing the frequencies of the majority class by the minority class. In this paper, we argue that, although imbalance-ratio is an informative measure for binary problems, it is not adequate for the multi-class scenario due to the fact that, in that scenario, it groups problems with disparate class-imbalance extents under the same numerical value. Thus, in order to overcome this drawback, in this paper, we propose imbalance-degree as a novel and normalised measure which is capable of properly measuring the class-imbalance extent of a multi-class problem. Experimental results show that imbalance-degree is more adequate than imbalance-ratio since it is more sensitive in reflecting the hindrance produced by skewed multi-class distributions to the learning processes.

Several popular graph embedding techniques for representation learning and dimensionality reduction rely on performing computationally expensive eigendecompositions to derive a nonlinear transformation of the input data space. The resulting eigenvectors encode the embedding coordinates for the training samples only, and so the embedding of novel data samples requires further costly computation. In this paper, we present a method for the out-of-sample extension of graph embeddings using deep neural networks (DNN) to parametrically approximate these nonlinear maps. Compared with traditional nonparametric out-of-sample extension methods, we demonstrate that the DNNs can generalize with equal or better fidelity and require orders of magnitude less computation at test time. Moreover, we find that unsupervised pretraining of the DNNs improves optimization for larger network sizes, thus removing sensitivity to model selection., Comment: 10 pages, 2 figures, 1 table, this paper is under consideration for publication in Pattern Recognition Letters

The wrapper feature selection method can achieve high classification accuracy. However, the cross-validation scheme of the wrapper method in evaluation phase is very expensive regarding computing resource consumption. In this paper, we propose a new statistical measure named as LW-index which could replace the expensive cross-validation scheme to evaluate the feature subset. Then, a new feature selection method, which is the combination of the proposed LW-index with Sequence Forward Search algorithm (SFS-LW), is presented in this paper. Further, we show through plenty of experiments conducted on nine UCI datasets that the proposed method can obtain similar classification accuracy as the wrapper method with centroid-based classifier or support vector machine, and its computation cost is approximate to the compared filter methods.

This paper presents an exploratory analysis of sex prediction from swipe gestures.Details of the software and data collection protocol are provided for reproducibility.The BestFirst feature selection technique has been analysed.Naive Bayes, logistic regression, SVM and decision tree classifiers have been tested.The results confirm the possibility of sex prediction from swipe gestures. Touch and multi-touch gestures are becoming the most common way to interact with technology such as smart phones, tablets and other mobile devices. The latest touch-screen input capacities have tremendously increased the quantity and quality of available gesture data, which has led to the exploration of its use in multiple disciplines from psychology to biometrics. Following research studies undertaken in similar modalities such as keystroke and mouse usage biometrics, the present work proposes the use of swipe gesture data for the prediction of soft-biometrics, specifically the user's sex. This paper details the software and protocol used for the data collection, the feature set extracted and subsequent machine learning analysis. Within this analysis, the BestFirst feature selection technique and classification algorithms (naive Bayes, logistic regression, support vector machine and decision tree) have been tested. The results of this exploratory analysis have confirmed the possibility of sex prediction from the swipe gesture data, obtaining an encouraging 78% accuracy rate using swipe gesture data from two different directions. These results will hopefully encourage further research in this area, where the prediction of soft-biometrics traits from swipe gesture data can play an important role in enhancing the authentication processes based on touch-screen devices.

We use RGB-D cameras data for foreground/background segmentation.Pixel level and region level background models based on color and depth data.Foreground region prediction, based on depth based histograms.Fusion of region based classifiers as mixture of experts. In the recent years, the computer vision community has shown great interest on depth-based applications thanks to the performance and flexibility of the new generation of RGB-D imagery. In this paper, we present an efficient background subtraction algorithm based on the fusion of multiple region-based classifiers that processes depth and color data provided by RGB-D cameras. Foreground objects are detected by combining a region-based foreground prediction (based on depth data) with different background models (based on a Mixture of Gaussian algorithm) providing color and depth descriptions of the scene at pixel and region level. The information given by these modules is fused in a mixture of experts fashion to improve the foreground detection accuracy. The main contributions of the paper are the region-based models of both background and foreground, built from the depth and color data. The obtained results using different database sequences demonstrate that the proposed approach leads to a higher detection accuracy with respect to existing state-of-the-art techniques.

International audience; This survey paper aims at providing a "literary" anthology of mathematical morphology on graphs. It describes in the English language many ideas stemming from a large number of different papers, hence providing a unified view of an active and diverse field of research.

Sequences of images from urban environments are increasing in number as well as their potential applications. They are being taken (from stationary and dynamic cameras) for applications such as traffic surveillance, or for autonomous driving, or for security applications, etc. The literature presents several different approaches for each application. For object detection, a common disadvantage is that they only consider images obtained from a stationary, or a dynamic, camera to train the detectors. This can lead to poor performances when the detectors are used in sequences of images from different types of cameras, or even a cross camera testing. e.g., training with data from a dynamic camera and testing with sequences from a stationary camera. Another disadvantage is that some approaches use several models for each point of view of the car, generating a lot of models and, in some cases, one classifier for each point of view. In this paper, we approach the problem of car detection using a model of the class car created with a dataset of static images and we use the model to detect cars in sequence of images that were collected from static and dynamic cameras, i.e., in a totally different setting than used for training. The creation of the model is done by an off-line learning phase, using an image database of cars in several points of view, PASCAL 2007. The model is based on a mixture of deformable part models that have been shown to give state of the art results for detection in static images. The results show that the proposed approach achieves better results than the state of the art approaches in sequence of images obtained from a stationary, or a dynamic camera. Another contribution of our paper is a ground truth of a large sequence of images available in the Internet.

Detection of anomalies is a broad field of study, which is applied in different areas such as data monitoring, navigation, and pattern recognition. In this paper we propose two measures to detect anomalous behaviors in an ensemble of classifiers by monitoring their decisions; one based on Mahalanobis distance and another based on information theory. These approaches are useful when an ensemble of classifiers is used and a decision is made by ordinary classifier fusion methods, while each classifier is devoted to monitor part of the environment. Upon detection of anomalous classifiers we propose a strategy that attempts to minimize adverse effects of faulty classifiers by excluding them from the ensemble. We applied this method to an artificial dataset and sensor-based human activity datasets, with different sensor configurations and two types of noise (additive and rotational on inertial sensors). We compared our method with two other well-known approaches, generalized likelihood ratio (GLR) and One-Class Support Vector Machine (OCSVM), which detect anomalies at data/feature level. We found that our method is comparable with GLR and OCSVM. The advantages of our method compared to them is that it avoids monitoring raw data or features and only takes into account the decisions that are made by their classifiers, therefore it is independent of sensor modality and nature of anomaly. On the other hand, we found that OCSVM is very sensitive to the chosen parameters and furthermore in different types of anomalies it may react differently. In this paper we discuss the application domains which benefit from our method.

In this paper, a novel illumination compensation method for face recognition under illumination variations is proposed. Rather than performing illumination compensation in a global way, the proposed method uses low-frequency discrete cosine transform (DCT) coefficients in the logarithm domain to estimate illumination in local areas. To estimate the illumination of every point more precisely, a mean operator is applied to refine the estimation. Experimental results on the CMU PIE database, the Yale Face database B and the Extended Yale Face database B demonstrate that the method is superior in comparison with other existing methods. Furthermore, a simplified version of the method is also proposed. Both theoretical analysis and experimental results demonstrate the validity and high computational efficiency of the simplified version. Performances of the proposed methods under different values of parameters are also discussed in the paper.

Security concerns regarding the stored biometric data is impeding the widespread public acceptance of biometric technology. Though a number of bio-crypto algorithms have been proposed, they have limited practical applicability due to the trade-off between recognition performance and security of the template. In this paper, we improve the recognition performance as well as the security of a fingerprint based biometric cryptosystem, called fingerprint fuzzy vault. We incorporate minutiae descriptors, which capture ridge orientation and frequency information in a minutia's neighborhood, in the vault construction using the fuzzy commitment approach. Experimental results show that with the use of minutiae descriptors, the fingerprint matching performance improves from an FAR of 0.7% to 0.01% at a GAR of 95% with some improvement in security as well. An analysis of security while considering two different attack scenarios is also presented. A preliminary version of this paper appeared in the International Conference on Pattern Recognition, 2008 and was selected as the Best Scientific Paper in the biometrics track.

Blob-like structures occur often in nature, where they aid in cueing and the pre-attentive process. These structures often overlap, form perceptual boundaries, and are heterogeneous in shape, size, and intensity. In this paper, voting, Voronoi tessellation, and level set methods are combined to delineate blob-like structures. Voting and subsequent Voronoi tessellation provide the initial condition and the boundary constraints for each blob, while curve evolution through level set formulation provides refined segmentation of each blob within the Voronoi region. The paper concludes with the application of the proposed method to a dataset produced from cell based fluorescence assays and stellar data.

A new computational model for active visual attention is introduced in this paper. The method extracts motion and shape features from video image sequences, and integrates these features to segment the input scene. The aim of this paper is to highlight the importance of the motion features present in our algorithms in the task of refining and/or enhancing scene segmentation in the method proposed. The estimation of these motion parameters is performed at each pixel of the input image by means of the accumulative computation method, using the so-called permanency memories. The paper shows some examples of how to use the ''motion presence'', ''module of the velocity'' and ''angle of the velocity'' motion features, all obtained from accumulative computation method, to adjust different scene segmentation outputs in this dynamic visual attention method.

This paper presents an effective dissimilarity measure for geometric graphs representing shapes. The proposed dissimilarity measure is a distance that combines a sparsification of the geometric graph based on the maximum diversity problem and a new node embedding that captures the topological neighborhood of nodes. The sparsification step aims to reduce the size of the graph and to correct the misdistribution of nodes on the geometric graph induced by the noise of image handling. Experimental evaluation shows that the sparsification algorithm retains the form of the shapes while decreasing the number of processed nodes which reduces the overall matching time. Furthermore, the proposed node embedding and similarity measure give better performance in comparison with existing graph matching approaches.

Multimodal teaching activity faces significant problems in Visual Question Answering (VQA), which involves simultaneous comprehension with reduced performance fidelity. However, Conventional methods are employed for portrayal and queries in a defined manner, which fails to accomplish the required performance accuracy rate. For elucidating the excellent image and question representation, this paper suggests an Adaptive Deep Concatenated Coder Framework (ADC–CF) that enrolls both the image and question attributes simultaneously with the optimized residual layer. The Coder Framework comprises of cascaded layers of Encoder-Decoder architecture, which captures rich, meaningful query characteristics and image details through the use of keywords employing significant object areas in the picture. ADC–CF layer has an encoder segment that blueprints the self-recognition of queries in which questions are concatenated to limit the answers and decoder segment blueprints the commanded-recognition of images. The simulation results of ADC–CF are tested with both the VQA datasets 1.0 and 2.0 and manifests an improved performance accuracy ratio of 72.45% for 1.0 dataset and 73.57% for 2.0 datasets, thus proving the reliability of the proposed framework.

The goal of this paper is to offer a framework for classification of images and video according to their "type", or "style"--a problem which is hard to define, but easy to illustrate; for example, identifying an artist by the Style of his/ her painting, or determining the activity in a video sequence. The paper offers a simple classification paradigm based on local properties of spatial or spatio-temporal blocks. The learning and classification are based on the naive Bayes classifier. A few experimental results are presented.

As a new pattern classification method, nearest feature line (NFL) provides an effective way to tackle the sort of pattern recognition problems where only limited data are available for training. In this paper, we explore the use of NFL for speaker identification in terms of limited data and examine how the NFL performs in such a vexing problem of various mismatches between training and test. In order to speed up NFL in decision-making, we propose an alternative method for similarity measure. We have applied the improved NFL to speaker identification of different operating modes. Its text-dependent performance is better than the dynamic time warping (DTW) on the Ti46 corpus, while its computational load is much lower than that of DTW. Moreover, we propose an utterance partitioning strategy used in the NFL for better performance. For the text-independent mode, we employ the NFL to be a new similarity measure in vector quantization (VQ), which causes the VQ to perform better on the KING corpus. Some computational issues on the NFL are also discussed in this paper.

This paper presents an optical music recognition (OMR) system that can automatically recognize the main musical symbols of a scanned paper-based music score. Two major stages are distinguished: the first one, using low-level pre-processing, detects the isolated objects and outputs some hypotheses about them; the second one has to take the final correct decision, through high-level processing including contextual information and music writing rules. This article exposes both stages of the method: after explaining in detail the first one, the symbol analysis process, it shows through first experiments that its outputs can efficiently be used as inputs for a high-level decision process.

Neural networks have achieved state of the art performance across a wide variety of machine learning tasks, often with large and computation-heavy models. Inducing sparseness as a way to reduce the memory and computation footprint of these models has seen significant research attention in recent years. In this paper, we present a new method for dynamic sparseness, whereby part of the computations are omitted dynamically, based on the input. For efficiency, we combined the idea of dynamic sparseness with block-wise matrix-vector multiplications. In contrast to static sparseness, which permanently zeroes out selected positions in weight matrices, our method preserves the full network capabilities by potentially accessing any trained weights. Yet, matrix vector multiplications are accelerated by omitting a pre-defined fraction of weight blocks from the matrix, based on the input. Experimental results on the task of language modeling, using recurrent and quasi-recurrent models, show that the proposed method can outperform static sparseness baselines. In addition, our method can reach similar language modeling perplexities as the dense baseline, at half the computational cost at inference time.

Deep metric learning maps visually similar images onto nearby locations and visually dissimilar images apart from each other in an embedding manifold. The learning process is mainly based on the supplied image negative and positive training pairs. In this paper, a dynamic sampling strategy is proposed to organize the training pairs in an easy-to-hard order to feed into the network. It allows the network to learn general boundaries between categories from the easy training pairs at its early stages and finalize the details of the model mainly relying on the hard training samples in the later. Compared to the existing training sample mining approaches, the hard samples are mined with little harm to the learned general model. This dynamic sampling strategy is formularized as two simple terms that are compatible with various loss functions. Consistent performance boost is observed when it is integrated with several popular loss functions on fashion search, fine-grained classification, and person re-identification tasks., Comment: 11 pages, 4 figures

In this paper, we propose a simple yet effective method to represent point clouds as sets of samples drawn from a cloud-specific probability distribution. This interpretation matches intrinsic characteristics of point clouds: the number of points and their ordering within a cloud is not important as all points are drawn from the proximity of the object boundary. We postulate to represent each cloud as a parameterized probability distribution of points in space, which is defined by a generative neural network. The network operates by composing several spatial transformations of point locations. Once trained, it provides a natural framework for point cloud manipulation. For instance we can decouple cloud shape from its orientation and provide routines for aligning a new cloud into a default spatial orientation. To exploit similarities between same-class objects and to improve model performance, we turn to weight sharing: networks that model densities of points belonging to objects in the same family share all parameters with the exception of a small, object-specific embedding vector. We show that these embedding vectors capture semantic relationships between objects. Our method leverages generative invertible flow networks to learn embeddings as well as to generate point clouds. Thanks to this formulation and contrary to similar approaches, we are able to train our model in an end-to-end fashion. As a result, our model offers competitive or superior quantitative results on benchmark datasets, while enabling unprecedented capabilities to perform cloud manipulation tasks, such as point cloud registration and regeneration, by a generative network.

Recently, great progress has been made in the field of Few-Shot Learning (FSL). While many different methods have been proposed, one of the key factors leading to higher FSL performance is surprisingly simple. It is the backbone network architecture used to embed the images of the few-shot tasks. While first works on FSL resorted to small architectures with just a few convolution layers, recent works show that large architectures pre-trained on the training portion of FSL datasets produce strong features that are more easily transferable to novel few-shot tasks, thus attaining significant gains to methods using them. Despite these observations, little to no work has been done towards finding the right backbone for FSL. In this paper we propose MetAdapt that not only meta-searches for an optimized architecture for FSL using Network Architecture Search (NAS), but also results in a model that can adaptively ‘re-wire’ itself predicting the better architecture for a given novel few-shot task. Using the proposed approach we observe strong results on two popular few-shot benchmarks: miniImageNet and FC100.

In this paper the analysis of gray level corner detection has been carried out. Performances of various cornerness measures are discussed with respect to four performances of robustness: detection, localization, stability and complexity. We have analyzed the interior differential features of the image surface of these cornerness measures. This paper presents a new approach called gradient-direction corner detector for the corner detection which is developed from the popular Plessey corner detection. The gradient-direction corner detector is based on the measure of the gradient module of the image gradient direction and the constraints of the false corner response suppression.

Collecting textual descriptions is an especially costly task for dense video captioning, since each event in the video needs to be annotated separately and a long descriptive paragraph needs to be provided. In this paper, we investigate a way to mitigate this heavy burden and propose to leverage captions of visually similar images as auxiliary context. Our model successfully fetches visually relevant images and combines noun and verb phrases from their captions to generating coherent descriptions. To this end, we use a generator and discriminator design, together with an attention-based fusion technique, to incorporate image captions as context in the video caption generation process. The experiments on the challenging ActivityNet Captions dataset demonstrate that our proposed approach achieves more accurate and more diverse video descriptions compared to the strong baseline using METEOR, BLEU and CIDEr-D metrics and qualitative evaluations.

To process data like text and speech, Natural Language Processing (NLP) is a valuable tool. As on of NLP’s upstream tasks, sequence labeling is a vital part of NLP through techniques like text classification, machine translation, and sentiment analysis. In this paper, our focus is on sequence labeling where we assign semantic labels within input sequences. We present two novel frameworks, namely SA-CRFLV-I and SA-CRFLV-II, that use latent variables within random fields. These frameworks make use of an encoding schema in the form of a latent variable to be able to capture the latent structure in the observed data. SA-CRFLV-I shows the best performance at the sentence level whereas SA-CRFLV-II works best at the word level. In our in-depth experimental results, we compare our frameworks with 4 well-known sequence prediction methodologies which include NER, reference parsing, chunking as well as POS tagging. The proposed frameworks are shown to have better performance in terms of many well-known metrics.

Automatic lesion detection from computed tomography (CT) scans is an important task in medical diagnosis. However, three frequent properties of medical data make CT lesion detection a challenging task: (1) Scale variance: Large scale variation is across lesion instances. Especially, it is extremely difficult to detect small lesions; (2) Imbalanced data: The data distributions are highly imbalanced, where few classes account for the majority of data; (3) Prediction stability: Based on our observations, an input lesion image with slightly pixel shift or translation can lead to drastic output mispredictions and this is not allowed for medical applications. To address these challenges, this paper proposes a Robust One-Stage Network (ROSNet) for robust CT lesion detection. Specifically, a novel nested structure of neural networks is developed to generate a series of feature pyramids for detecting CT lesions in various scales, an effective data sensitive class-balanced loss as well as a shift-invariant downsampling strategy are also introduced to improve the detection performance. Experiments are conducted on a large-scale and diverse dataset, DeepLesion, showing that ROSNet outperforms the best performance in MICCAI 2019 by 3.95% (2-class detection task) and 25.41% (8-class detection task) in terms of mean average precision (mAP).

An adaptive method based on three dimensional Gaussian Markov Random fields (3D-GMRF) is proposed in this paper for volumetric texture segmentation. A feature vector is extracted for each voxel in a given volumetric texture image using an estimation cube. However, the selection of the size for this estimation cube causes some fundamental issues related to the uncertainty principle and the inability of the model to capture different texture patterns. These issues are tackled here by employing an adaptive method where the size of the estimation cube is adaptively varying to capture different patterns and also minimize the number of voxels that are related to different texture classes inside the estimation cube. The feature vectors that consist of the estimated parameters of the GMRF and form the parameter volume are hence employed to segment volumetric textures. These features are smoothed by applying an averaging filter using an adaptive averaging technique. Such an averaging filter improves the segmentation results considerably. Our method proposed here is evaluated on a synthetic volumetric texture dataset and compared with other methods to demonstrate the superiority of our segmentation method.

The pulse transit time (PTT) can provide valuable insight into cardiovascular health, specifically regarding arterial stiffness and blood pressure. Traditionally, PTT is derived by calculating the time difference between two photoplethysmography (PPG) measurements, which require a set of body-worn sensors attached to the skin. Recently, remote photoplethysmography (rPPG) has been proposed as a contactless monitoring alternative. The main problem with rPPG based PTT estimation is that motion artifacts affect the shape of waveform leading to the shift or over-detected peaks, which decreases the accuracy of PTT. To overcome this problem, this paper presents a robust pulse-by-pulse PTT estimation framework using a signal quality metric. By exploiting the local temporal information and global periodic characteristics, the metric automatically assesses pulse quality of signal on a pulse-by-pulse basis, and calculates the probabilities of the pulse peak being the actual peak. Furthermore, in order to cope with over-detected and shift pulse peaks, Kalman filter complemented by the proposed signal quality metric is used to adaptively adjust the peaks based on the estimated probability. All the refined peaks are finally used for pulse-by-pulse PTT estimation. The experiment results are promising, suggesting that the proposed framework provides a robust and more accurate PTT estimation in real applications.

Small object detection in crowded scene aims to find those tiny targets with very limited resolution from crowded scenes. Due to very little information available on tiny objects, it is often not suitable to detect them merely based on the information presented inside their bounding boxes, resulting low accuracy. In this paper, we propose to exploit the semantic similarity among all predicted objects’ candidates to boost the performance of detectors when handling tiny objects. For this purpose, we construct a pairwise constraint to depict such semantic similarity and propose a new framework based on Discriminative Learning and Graph-Cut techniques. Experiments conducted on three widely used benchmark datasets demonstrate the improvement over the state-of-the-art approaches gained by applying this idea.

[EN] Alpha integration is a family of integrators that encompasses many classic fusion operators (e.g., mean, product, minimum, maximum) as particular cases. This paper proposes vector score integration (VSI), a new alpha integration method for late fusion of multiple classifiers considering the joint effect of all the classes of the multi-class problem. Theoretical derivations to optimize the parameters of VSI for achieving the minimum probability of error are provided. VSI was applied to two classification tasks using electroencephalographic signals. The first task was the automatic stage classification of a neuropsychological test performed by epileptic subjects and the second one was the classification of sleep stages from apnea patients. Four single classifiers (linear and quadratic discriminant analysis, naive Bayes, and random forest) and three competitive fusion methods were estimated for comparison: mean, majority voting, and separated score integration (SSI). SSI is based on alpha integration, but unlike the proposed method, it considers the scores from each class in isolation, not accounting for possible dependencies among scores corresponding to different classes. VSI was able to optimally combine the results from all the single classifiers, in terms of accuracy and kappa coefficient, and outperformed the results of the other fusion methods in both applications., This work was supported by the Spanish Administration and European Union under grant TEC2017-84743-P.

Nowadays, it is common in Historical Demography the use of individual-level data as a consequence of a predominant life-course approach for the understanding of the demographic behaviour, family transition, mobility, etc. Advanced record linkage is key since it allows increasing the data complexity and its volume to be analyzed. However, current methods are constrained to link data from the same kind of sources. Knowledge graph are flexible semantic representations, which allow to encode data variability and semantic relations in a structured manner. In this paper we propose the use of knowledge graph methods to tackle record linkage tasks. The proposed method, named WERL, takes advantage of the main knowledge graph properties and learns embedding vectors to encode census information. These embeddings are properly weighted to maximize the record linkage performance. We have evaluated this method on benchmark data sets and we have compared it to related methods with stimulating and satisfactory results.

A method for speeding-up data projections onto kernel-induced feature spaces (derived using e.g. kernel Principal Component Analysis - kPCA) is presented in the paper. The proposed idea is to simplify the derived features, implicitly defined by all training samples and dominant eigenvectors of problem-specific generalized eigenproblems, by appropriate approximations. Instead of employing the whole training set, we propose to use a small pool of its appropriately selected representatives and we formulate a rule for deriving the corresponding weight vectors that replace the considered dominant eigenvectors. The representatives are determined via clustering of training data, whereas weighting coefficients are chosen to minimize original feature approximation errors. The concept has been experimentally verified for kernel-PCA using both artificial and real datasets. It has been shown that the presented approach provides reduction in feature-extraction complexity, which implies a proportional increase in data projection speed, by one-to-two orders of magnitude, without sacrificing data analysis accuracy. Therefore, the proposed approach is well-suited for kernel-based, intelligent data analysis applications that are to be executed on resource-limited systems, such as embedded or IoT devices, or for systems where processing time is critical.

In this paper, we propose a novel method for mild cognitive impairment detection based on jointly exploiting the complex network and the neural network paradigm. In particular, the method is based on ensembling different brain structural “perspectives” with artificial neural networks. On one hand, these perspectives are obtained with complex network measures tailored to describe the altered brain connectivity. In turn, the brain reconstruction is obtained by combining diffusion-weighted imaging (DWI) data to tractography algorithms. On the other hand, artificial neural networks provide a means to learn a mapping from topological properties of the brain to the presence or absence of cognitive decline. The effectiveness of the method is studied on a well-known benchmark data set in order to evaluate if it can provide an automatic tool to support the early disease diagnosis. Also, the effects of balancing issues are investigated to further assess the reliability of the complex network approach to DWI data.

After the discovery of adversarial examples and their adverse effects on deep learning models, many studies focused on finding more diverse methods to generate these carefully crafted samples. Although empirical results on the effectiveness of adversarial example generation methods against defense mechanisms are discussed in detail in the literature, an in-depth study of the theoretical properties and the perturbation effectiveness of these adversarial attacks has largely been lacking. In this paper, we investigate the objective functions of three popular methods for adversarial example generation: the L-BFGS attack, the Iterative Fast Gradient Sign attack, and Carlini & Wagner's attack (CW). Specifically, we perform a comparative and formal analysis of the loss functions underlying the aforementioned attacks while laying out large-scale experimental results on ImageNet dataset. This analysis exposes (1) the faster optimization speed as well as the constrained optimization space of the cross-entropy loss, (2) the detrimental effects of using the signature of the cross-entropy loss on optimization precision as well as optimization space, and (3) the slow optimization speed of the logit loss in the context of adversariality. Our experiments reveal that the Iterative Fast Gradient Sign attack, which is thought to be fast for generating adversarial examples, is the worst attack in terms of the number of iterations required to create adversarial examples in the setting of equal perturbation. Moreover, our experiments show that the underlying loss function of CW, which is criticized for being substantially slower than other adversarial attacks, is not that much slower than other loss functions. Finally, we analyze how well neural networks can identify adversarial perturbations generated by the attacks under consideration, hereby revisiting the idea of adversarial retraining on ImageNet., Accepted for publication in Pattern Recognition Letters, 2020

An activation function has crucial role in a deep neural network. A simple rectified linear unit (ReLU) are widely used for the activation function. In this paper, a weighted sigmoid gate unit (WiG) is proposed as the activation function. The proposed WiG consists of a multiplication of inputs and the weighted sigmoid gate. It is shown that the WiG includes the ReLU and same activation functions as a special case. Many activation functions have been proposed to overcome the performance of the ReLU. In the literature, the performance is mainly evaluated with an object recognition task. The proposed WiG is evaluated with the object recognition task and the image restoration task. Then, the expeirmental comparisons demonstrate the proposed WiG overcomes the existing activation functions including the ReLU.

Although Zero Shot Learning (ZSL) has attracted more and more attention due to its powerful ability of recognizing new objects without retraining, it has a serious drawback that it only focuses on unseen classes during prediction. To solve this issue, Generalized ZSL (GZSL) extends the search range to both seen and unseen classes, which makes it a more realistic and challenging task. Conventional methods on GZSL often suffer from the domain shift problem on seen classes because they have only seen data for training. Deep Calibration Network (DCN) tries to minimize the entropy of assigning seen data to unseen classes to balance the training on both seen and unseen classes. However, there are still two problems for DCN, one is the hubness problem and another is the lack of training guidance. In this paper, to solve the two problems, we propose a novel method called PSeudo Distribution (PSD), which exploits the attribute similarity between seen classes and unseen classes as the training guidance to assign the seen data to unseen classes. In addition, the attribute similarity is also compressed to one-hot vector to further encourage the certainty of the model. Besides, the visual space is utilized as the embedding space, which can well settle the hubness problem. Extensive experiments are conducted on four popular datasets, and the results show the superiority of the proposed method.

Automated chest X-ray (CXR) image analysis is often subject to serious disruption and misguided by its imaging artifacts and noise regions. To minimize such negative effects, many state-of-the-art works have made great efforts in the precise segmentation of lung fields. Based on these works, some image-based features can be extracted directly from lung filed to provide clues for many types of lung diseases such as lung nodule, cardiomegaly, pneumothorax, or emphysema. In this paper, we propose a novel two-stream collaborative network call TSCN for multi-label CXR image classification based on lung segmentation. Specifically, we first train a robust lung segmentor with U-Net and apply it to capture the lung filed from the original CXR image. Then we perform a two-stream feature fusion operation to aggregate the contextual information in both the global image and lung field for complementary feature learning. By taking advantage of the two-stream deep structures and two types of image inputs, a novel self-adaptive weighted fusion scheme is designed to jointly learn these two feature streams in the end-to-end training phase, in order to realize adaptive two-stream feature subsets selection and optimization. Extensive experiments on the ChestX-ray14 dataset demonstrate the effectiveness of the proposed method as compared with the state-of-the-art baselines.