Search

Multilabel classification as a data mining task has recently attracted increasing interest from researchers. Many current data mining applications address problems with instances that belong to more than one class. These problems require the development of new, efficient methods. Advantageously using the correlation among different labels can provide better performance than methods that manage each label separately. In recent decades, many methods have been developed to deal with multilabel datasets, which makes it difficult to decide which method is the most appropriate for a given task. In this paper, we present the most comprehensive comparison carried out so far. We compare a total of 62 different methods and several configurations of each one for a total of 197 trained models. We also use a large set of problems comprising 65 datasets. In addition, we studied the efficiency of the methods considering six different classification performance metrics. Our results show that, although there are methods that repeatedly appear among the top-performing models, the best methods are closely related to the metric used for evaluating the performance. We also analyzed different aspects of the behavior of the methods. • We present a thorough comparison of multi-label methods. • The behavior depending on base classifiers is studied for many methods. • Six different metrics are used in the comparison. • Clustering of similar methods is carried out. • The behavior of the methods depending characteristics is carried out. [ABSTRACT FROM AUTHOR]

Deep learning techniques have been widely used for two-dimensional (2D) and three-dimensional (3D) computer vision problems, such as object detection, super-resolution (SR) and classification to name a few. Radar images suffer from poor resolution as compared to optical images, hence developing a high-accuracy model to solve computer vision problems, such as a classifier, is a challenge. This is because of the lack of high-frequency details in the input images which makes it difficult for the classifier model to generate accurate predictions. Ways of addressing this challenge include training the learning model with a large dataset or using a more complicated model, such as deeper layer architecture. However, employing such approaches might result in the overfitting of the model, where the model might not generalize well on previously unseen data. Also, generating a large dataset for training the model is a challenging task, especially in the case of radar images. An alternate solution for achieving high accuracy in radar classification problems is provided in this paper wherein a CNN-enabled super-resolution (SR) model is integrated with the classifier model. The SR model is trained to generate high-resolution (HR) millimeter-wave (mmW) images from any input low-resolution (LR) mmW images. These resolved images from the SR model will be used by the classifier model to classify the input images into appropriate classes, consisting of threat and non-threat objects. The training data for the dual CNN layers are generated using a numerical model of a near-field coded-aperture computational imaging (CI) system. This trained dual CNN model is tested with simulated data generated from the CI numerical model wherein a high classification accuracy of 95% and a fast inference time of 0.193 s are obtained, making it suitable for real-time automated threat classification applications. For fair comparison, the developed CNN model is also validated with experimentally generated reconstruction data, in which case, a classification accuracy of 94% is obtained. • An integrated convolutional neural network is developed which jointly performs super-resolution and classification tasks. The super-resolution part enhances the resolution of millimeter-wave input images and the classifier part predicts a class for the enhanced image. • The super-resolution part of the model enhances both the cross-range and range resolutions of the input images. In addition, the model uses both the real and imaginary parts of the reconstruction data to perform the super-resolution task. • The super-resolution and the classifier models are trained on synthesized data instead of experimentally generated data. To offer a realistic approach, additive Gaussian noise is incorporated into the synthesized data. • Validation of the integrated model with both synthesized and experimental data shows a real-time accurate super-resolution and multi-label classification performance. [ABSTRACT FROM AUTHOR]

Node classification is a popular and challenging task in graph neural networks, and existing approaches are mainly developed for a single network. With the advances in domain adaptation, researchers tend to leverage knowledge extracted from a fully-labeled source network to further improve the node classification performance in an unlabeled target network. This learning paradigm refers to cross-network node classification, which is the topic we studied in this paper. Specifically, we propose a novel model named Transferable Graph Auto-Encoders (TGAE), which first encodes the initial network data into latent representations and then decodes the learned features to preserve graph information. In the encoding phase, TGAE adopts the attentional mechanism to fuse the local and global information of nodes to discover latent node representations. To obtain transferable features between the source and target networks, TGAE aligns their distributions based on the learned representations by reducing marginal and conditional distribution differences. In the decoding phase, the latent representations are subjected to pairwise and reconstruction constraints, thus preserving structural proximity and graph topology information to learn discriminative features. Besides, a node classifier is trained to enhance the discriminant of the node representations further. Experimental results on several real-world datasets demonstrate that the proposed model achieves state-of-the-art performance in cross-network node classification tasks compared with existing methods. • TGAE is proposed to handle cross-network node classification problem. • TGAE encodes local and global information and preserves structural graph information. • Experimental results demonstrate the superior performance of the proposed model. [ABSTRACT FROM AUTHOR]

Multi-view learning has demonstrated promising performance for 3D shape recognition. However, existing multi-view methods usually focus on fusing multiple views and ignore the structural and discriminative information carried by 2D views. In this paper, we propose a multi-view hierarchical self-attention network (MHSAN) to explore the geometric and discriminative information from complex 2D views. Specifically, MHSAN consists of two self-attention networks. First, a global self-attention network is adopted to exploit the structure information by embedding position information of views. Then, the discriminative self-attention network learns discriminative information from the views with high classification scores. Through the proposed MHSAN, the geometric and discriminative information is condensed as the novel representation of 3D shapes. To validate the effectiveness of our proposed method, extensive experiments have been conducted on three 3D shape benchmarks. Experimental results demonstrate that our method is generally superior to the state-of-the-art methods in 3D shape classification and retrieval tasks. • A hierarchical self-attention network for 3D shape recognition is proposed. • The global information and the discriminative information are extracted. • Position information helps to learn the structure information of 3D object. • The views with high discriminative scores are given special attention. [ABSTRACT FROM AUTHOR]

Time series classification using phase-independent subsequences called shapelets is one of the best approaches in the state of the art. This approach is especially characterized by its interpretable property and its fast prediction time. However, given a dataset of n time series of length at most m , learning shapelets requires a computation time of O (n 2 m 4) which is too high for practical datasets. In this paper, we exploit the fact that shapelets are shared by the members of the same class to propose the SAST (Scalable and Accurate Subsequence Transform) algorithm which has a time complexity of O (n m 3). SAST is accurate, interpretable and does not learn redundant shapelets. The experiments we conducted on the UCR archive datasets showed that SAST is more accurate than the state of the art Shapelet Transform algorithm while being significantly more scalable. • We introduce the core shapelet recognition task. • We claim that time series classification by shapelets is a core shapelet recognition task. • We propose the SAST method to successfully performs the core shapelet recognition task in O (n m 3). • SAST is competive and more scalable than the SOTA shapelet methods on the UCR archive. • The implementation of our method is publicly accessible on GitHub. [ABSTRACT FROM AUTHOR]

In real applications, label noise has a great influence on data modeling. As one kind of label noise treatment method, noise filter has attracted extensive attention recently. The existing filters perform well in dealing with label noise completely at random (NCAR) but poorly in dealing with the label noise in the form of clusters (LLC). Besides, the existing filters may over remove the samples located at the classification boundaries, thereby affecting the generalization performance of classifiers. To fill these gaps, we propose a general elevating framework for label noise filters. The core idea of the framework is to improve the filtering performance of the existing filters by sample reduction. Specifically, since most of the existing filters are based on classifier prediction, and the complexity of samples at the boundary will affect the prediction performance of classifiers. Therefore, reducing the complexity of the boundary sample is very helpful to improve the performance of the filters. To this end, we propose a sample reduction method, which can not only reduce the complexity of the sample at the boundary but also convert LLC to NCAR, to get some representative samples. Next, the filter based on classifier prediction is employed to recognize the noisy representative samples. Finally, the noisy labeled samples in the given data set are found according to the identified noisy representatives. Furthermore, through empirical analysis, we found that compared with some classical metrics for evaluating the performance of noise filters, classification accuracy is more suitable to measure the performance of filters. Exhaustive experiments testify the validity of the framework, and the experimental results demonstrate that the performance of our framework is especially outstanding for LLC treatment. • The proposed noise filtering framework can alleviate the phenomenon of over-cleaning. • The performance of most existing noise filters can be improved with this framework. • The metric recommended in this paper can guide the research of the noise filter. [ABSTRACT FROM AUTHOR]

Deep Neural Networks have received considerable attention in recent years. As the complexity of network architecture increases in relation to the task complexity, it becomes harder to manually craft an optimal neural network architecture and train it to convergence. As such, Neural Architecture Search (NAS) is becoming far more prevalent within computer vision research, especially when the construction of efficient, smaller network architectures is becoming an increasingly important area of research, for which NAS is well suited. However, despite their promise, contemporary and end-to-end NAS pipeline require vast computational training resources. In this paper, we present a comprehensive overview of contemporary NAS approaches with respect to image classification, object detection, and image segmentation. We adopt consistent terminology to overcome contradictions common within existing NAS literature. Furthermore, we identify and compare current performance limitations in addition to highlighting directions for future NAS research. [ABSTRACT FROM AUTHOR]

Decision tree algorithm, because of its strong interpretability and high algorithm efficiency, is widely used in the field of pattern recognition and classification. When the number of data samples is small and there is uncertainty in the data, it is difficult for the traditional decision tree algorithm to fully mine the effective information in the data. In this paper, we use the Dempster–Shafer framework to model data uncertainty and propose a hierarchical interval estimation method to improve decision tree algorithms. The proposed method constructs intervals through two methods of attribute boundary and mean square error estimation, which not only utilizes the characteristics of intervals to model the inaccuracy of data, but also constrains intervals from two aspects, narrowing the representation range of available information. By comparing with the classic decision tree algorithm and the decision tree algorithm based on single interval estimation, the proposed method can perform classification tasks robustly and accurately in different types of data under seven data sets. • We propose an evidential decision tree based on a reliability-based BPA generation method and hierarchical interval estimation. • The proposed method greatly reduces the influence of the labels of uncertain data on the classification accuracy of the decision tree and strengthens the robustness of the algorithm. • Comparison with fifteen different decision trees, the accuracy of our proposed method on seven datasets is higher than other methods especially in the face of with uncertain attributes and labels. [ABSTRACT FROM AUTHOR]

Plenty of models have been presented to handle the hypergraph node classification. However, very few of these methods consider contrastive learning, which is popular due to its great power to represent instances. This paper makes an attempt to leverage contrastive learning to hypergraph representation learning. Specifically, we propose a novel method called Collaborative Contrastive Learning (CCL), which incorporates a generated standard graph with the hypergraph. The main technical contribution here is that we develop a collaborative contrastive schema, which performs contrast between the node views obtained from the standard graph and hypergraph in each network layer, thus making the contrast collaborative. To be precise, in the first layer, the view from the standard graph is used to augment that from the hypergraph. Then, in the next layer, the augmented features are adopted to train a new representation to augment the view from the standard graph conversely. With this setting, the learning procedure is alternated between the standard graph and hypergraph. As a result, the learning on the standard graph and hypergraph is collaborative and leads to the final informative node representation. Experimental results on several widely used datasets validate the effectiveness of the proposed model. • CCL is proposed to handle hypergraph node classification problem. • CCL considers the contrast in each layer from GCN and HGCN. • CCL exploits the convolutional networks on the standard graph and hypergraph. • Experimental results demonstrate the superior performance of the proposed model. [ABSTRACT FROM AUTHOR]

• This paper solves the problem of orthogonal linear discriminant analysis (OLDA) from the novel viewpoint of scatter matrix orthogonal decomposition. • The method can obtain approximately orthogonal sparse discriminative vectors for dimensionality reduction and jointly sparse feature extraction. • Theoretical analysis shows that OLDA can be derived by the constrained scatter matrix decomposition. • The method outperforms several well-known LDA-based and sparse learning methods on four data sets (i.e., COIL100, USPS, ICADAR2003 and CMU PIE). Orthogonal Linear Discriminant Analysis (OLDA) based on generalized Eigen-equation is widely used in the field of computer vision and pattern recognition. However, the performance of OLDA for feature extraction and classification needs to be improved as it lacks sparsity for better interpretation of the features. Moreover, computing the orthogonal sparse projections based on LDA is very difficult and is still unsolved. To solve these problems, in this paper, we propose a method called Jointly Sparse Orthogonal Linear Discriminant Analysis (JSOLDA). Different from the existing OLDA, JSOLDA is proposed from a novel viewpoint of scatter matrix decomposition. Theoretical analysis shows that OLDA can be derived by the constrained scatter matrix decomposition. In addition, by imposing L 2,1 -norm on the penalty term, the proposed JSOLDA can obtain the jointly sparse orthogonal projections to perform feature extraction. We also design an iterative algorithm to obtain the optimal solution. Systematic theoretical analysis between the OLDA and JSOLDA are uncovered. Both of convergence and computational complexity are also discussed. Experimental results on four data sets (i.e., COIL100, USPS, ICADAR2003 and CMU PIE) indicate that JSOLDA outperforms several well-known LDA-based and L 2,1 -norm based methods. [ABSTRACT FROM AUTHOR]

• We proposed the method of constructing a fuzzy granular recurrence plot (FGRP method). • We designed an FGRP method based classification model. • The experiment shows the good performance of the FGRP method in reducing the influence of noise. • The experiment presents the good performance of the FGRP method based classification model. Recently, recurrence plot (RP) and its quantification techniques have become an important research tool in nonlinear analysis. In the existing researches, an RP is directly established on a time series ignoring the influence of noise on data, which will affect our judgement on the dynamic properties of a system. To tackle the problem there, this paper proposes a novel recurrence plot, namely fuzzy granular recurrence plot (FGRP). An FGRP of a time series is built not directly on the time series itself but on its corresponding granular time series which is composed of fuzzy information granules. With specific capability, fuzzy information granules are used as building blocks of an FGRP to achieve high-level, compact and understandable signal models. In order to apply the FGRP method to time series classification tasks, an FGRP based classification model is designed in this paper. Subsequent experiments show that the FGRP of a time series can reduce the effect of noise, and the FGRP based classification model can improve the classification performance. [ABSTRACT FROM AUTHOR]

Recent graph neural networks (GNN) has achieved remarkable performance in node representation learning. One key factor of GNN's success is the smoothness property on node representations. Despite this, most GNN models are fragile to the perturbations on graph inputs and could learn unreliable node representations. In this paper, we study how to learn node representations against perturbations in GNN. Specifically, we consider that a node representation should remain stable under slight perturbations on the input, and node representations from different structures should be identifiable, which two are termed as the stability and identifiability on node representations, respectively. To this end, we propose a novel model called Stability-Identifiability GNN Against Perturbations (SIGNNAP) that learns reliable node representations in an unsupervised manner. SIGNNAP formalizes the stability and identifiability by a contrastive objective and preserves the smoothness with existing GNN backbones. The proposed method is a generic framework that can be equipped with many other backbone models (e.g. GCN, GraphSage and GAT). Extensive experiments on six benchmarks under both transductive and inductive learning setups of node classification demonstrate the effectiveness of our method. Codes and data are available online: https://github.com/xuChenSJTU/SIGNNAP-master-online [ABSTRACT FROM AUTHOR]

In this paper we propose a new version of penalty-based aggregation functions, the Multi Cost Aggregation choosing functions (MCAs), in which the function to minimize is constructed using a convex combination of two relaxed versions of restricted equivalence and dissimilarity functions instead of a penalty function. We additionally suggest two different alternatives to train a MCA in a supervised classification task in order to adapt the aggregation to each vector of inputs. We apply the proposed MCA in a Motor Imagery-based Brain–Computer Interface (MI-BCI) system to improve its decision making phase. We also evaluate the classical aggregation with our new aggregation procedure in two publicly available datasets. We obtain an accuracy of 82.31% for a left vs. right hand in the Clinical BCI challenge (CBCIC) dataset, and a performance of 62.43% for the four-class case in the BCI Competition IV 2a dataset compared to a 82.15% and 60.56% using the arithmetic mean. Finally, we have also tested the goodness of our proposal against other MI-BCI systems, obtaining better results than those using other decision making schemes and Deep Learning on the same datasets. • We develop relaxed versions of Restricted Equivalence Functions. • We aggregate information from different classifiers trained on different wave bands. • The new aggregations learn from the input data to discriminate between classes. • The novel aggregation functions can be adapted to the original data. [ABSTRACT FROM AUTHOR]

Long-tailed learning is attracting increasing attention due to the unbalanced distributions of real-world data. The aim is to train well-performing depth models. Traditional knowledge transfer methods for long-tailed learning are classified into feature-based horizontal knowledge transfer (HKT) and class-based vertical knowledge transfer (VKT). HKT transfers head-to-tail feature knowledge from different classes to improve classification performance when there are few tail classes. However, HKT easily leads to invalid transfer due to the deviation caused by the difference between the knowledge of head and tail classes. Fortunately, the class space has a multi-grained relationship and can form a multi-granularity knowledge graph (MGKG), which can be recast as coarse-grained and fine-grained losses to guide VKT. In this paper, we propose a hierarchical long-tailed classification method based on multi-granularity knowledge transfer (MGKT), which vertically transfers knowledge from coarse- to fine-grained classes. First, we exploit the semantic information of classes to construct an MGKG, which forms an affiliation of fine- and coarse-grained classes. Fine-grained knowledge can inherit coarse-grained knowledge to reduce transfer bias with the help of MGKG. We then propose a multi-scale feature fusion network, which aims to fully mine the rich information of the features to drive MGKT. Experiments show that the proposed model outperforms several state-of-the-art models in classifying long-tailed data. For example, our model performed 4.46% better than the next-best model on the SUN-LT dataset. • We propose a multi-scale feature fusion network about channel and spatial features. • We investigate a multi-granularity relationship of class space. • We explore a vertical transfer of coarse- to fine-grained knowledge. [ABSTRACT FROM AUTHOR]

• We propose novel graph classification method for better classification performance. • We develop an edge feature scheme and an add-on layer for discriminative feature learning. • We create corresponding graph datasets based upon ModelNet40, ModelNet10 and ShapeNet Part datasets and will release them to the community. Spectral graph convolutional neural networks (GCNNs) have been producing encouraging results in graph classification tasks. However, most spectral GCNNs utilize fixed graphs when aggregating node features while omitting edge feature learning and failing to get an optimal graph structure. Moreover, many existing graph datasets do not provide initialized edge features, further restraining the ability of learning edge features via spectral GCNNs. In this paper, we try to address this issue by designing an edge feature scheme and an add-on layer between every two stacked graph convolution layers in spectral GCNN. Both are lightweight while effective in filling the gap between edge feature learning and performance enhancement of graph classification. The edge feature scheme makes edge features adapt to node representations at different spectral graph convolution layers. The add-on layer helps adjust the edge features to an optimal graph structure. To test the effectiveness of our method, we take Euclidean positions as initial node features and extract graphs with semantic information from point cloud objects. The node features of our extracted graphs are more scalable for edge feature learning than most existing graph datasets (in one-hot encoded label format). Three new graph datasets are constructed based on ModelNet40, ModelNet10 and ShapeNet Part datasets. Experimental results show that our method outperforms state-of-the-art graph classification methods on the new datasets. Our code and the constructed graph datasets will be released to the community. [ABSTRACT FROM AUTHOR]

• A novel method CIDA, which uses the independence/dependence between features and the corresponding class/domain labels, is proposed to deal with UDA problem. • The objective function can be embedded into deep network architectures, and the whole model can be trained in an end-to-end manner. • A probabilistic interpretation of CIDA with Gaussian process is presented. • The effectiveness of CIDA is validated by several UDA benchmark datasets. Learning domain-adaptive features is important to tackle the dataset bias problem, where data distributions in the labeled source domain and the unlabeled target domain can be different. The critical issue is to identify and then reduce the redundant information including class-irrelevant and domain-specific features. In this paper, a conditional independence induced unsupervised domain adaptation (CIDA) method is proposed to tackle the challenges. It aims to find the low-dimensional and transferable feature representation of each observation, namely the latent variable in the domain-adaptive subspace. Technically, two mutual information terms are optimized at the same time. One is the mutual information between the latent variable and the class label, and the other is the mutual information between the latent variable and the domain label. Note that the key module can be approximately reformulated as a conditional independence/dependence based optimization problem, and thus, it has a probabilistic interpretation with the Gaussian process. Temporary labels of the target samples and the model parameters are alternatively optimized. The objective function can be incorporated with deep network architectures, and the algorithm is implemented iteratively in an end-to-end manner. Extensive experiments are conducted on several benchmark datasets, and the results show effectiveness of CIDA. [ABSTRACT FROM AUTHOR]

• Discrete network embedding (DNE) is proposed for compact representation. DNE leverages hash code to represent node, and dramatically reduces computational and storage costs. • Deep discrete attributed network embedding (DDANE) is proposed to effectively leverage node attribute and network structure in attributed network. The proposed DDANE trains an improved graph convolutional network autoencoder to encode node attribute and network structure into latent discrete embedding. • Extensive experiments demonstrate the proposed methods exhibit lower storage and computational complexity than state-of-the-art network embedding methods, and achieve satisfactory accuracy. Network embedding has shown promising performance in real-world applications. The network embedding typically lies in a continuous vector space, where storage and computation costs are high, especially in large-scale applications. This paper proposes more compact representation to fulfill the gap. The proposed discrete network embedding (DNE) leverages hash code to represent node in Hamming space. The Hamming similarity between hash codes approximates the ground-truth similarity. The embedding and classifier are jointly learned to improve compactness and discrimination. The proposed multi-class classifier is further constrained to be discrete to expedite classification. In addition, this paper further extends DNE and proposes deep discrete attributed network embedding (DDANE) to learn compact deep embedding from more informative attributed network. From the perspective of generalized signal smoothing, the proposed DDANE trains an improved graph convolutional network autoencoder to effectively leverage node attribute and network structure. Extensive experiments on node classification demonstrate the proposed methods exhibit lower storage and computational complexity than state-of-the-art network embedding methods, and achieve satisfactory accuracy. [ABSTRACT FROM AUTHOR]

• We propose a big and real RGB 3D hand pose estimation dataset that contains the 2D and 3D coordinates of the joints, status of the joints occlusion, classes of different gestures, and segmentation of the 15 parts of the hand. • We propose semantic segmentation of the hand parts for estimating the weight of the occlusion at hand joints. • We exploit the occlusion information for estimating the 3D hand pose from a single RGB image. • The proposed framework is a hybrid method of classification and estimation networks to increase the validity and the accuracy of the predicted hand poses. • Classification of the hand poses is based on the different aspects including gesture, the direction of the palm, and direction of the hand. In this paper, a new framework for 3D hand pose estimation using a single RGB image is proposed. The framework is composed of two blocks. The first block formulates the hand pose estimation as a classification problem. Since the human hand can perform numerous poses, the classification network needs a huge number of parameters. So, we propose to classify hand poses based on three different aspects, including hand gesture, hand direction, and palm direction. In this way, the number of parameters will be significantly reduced. The motivation behind the classification block is that the model deals with the image as a whole and extracts global features. Furthermore, the output of the classification model is a valid pose that does not include any unexpected angle at joints. The second block estimates the 3D coordinates of the hand joints and focuses more on the details of the image pattern. RGB-based 3D hand pose estimation is an inherently ill-posed problem due to the lack of depth information in the 2D image. We propose to use the occlusion status of the hand joints to solve this problem. The occlusion status of the joints has been labeled manually. Some joints are partially occluded, and we propose to compute the extent of the occlusion by semantic segmentation. The existing methods in this field mostly used synthetic datasets. But all the models proposed in this paper are trained on more than 50 K real images. Extensive experiments on our new dataset and two other benchmark datasets show that the proposed method can achieve good performance. We also analyze the validity of the predicted poses, and the results show that the classification block increases the validity of the poses. [ABSTRACT FROM AUTHOR]

• A data classification method which combines phase-field with DNN is proposed. • A MNIST digital database is adopted as the experimental subjects. • The tolerance value is set to judge whether the output value in DNN is abnormal. • The proposed method performs excellent in both binary and multi-classification. • The proposed method performs better than single DNN model or phase-field model. This paper proposes a novel classification method. Firstly, we use the deep neural network (DNN) to classify the training set. After several iterations, we obtain the output vector Y. The component of the largest value in vector Y is represented as the label being classified, which we take as the output value. Because we chose the sigmoid function as our activation function, the output value is between 0 and 1. Therefore, the output value can represents the probability of the classified label by the DNN. Depending on the distribution of output values, we set tolerance values (T o l) that categorize similar output values as the same label in the DNN. If the output value is lower than T o l , we consider it categorically anomalous. Subsequently, we use the Phase-Field model to classify these anomalies and obtain better classification results. As this classification method combines Phase-Field model and DNN, we named it Phase-Field-DNN. In the numerical experiment using MNIST handwritten digit data set as experimental data, the classification accuracy of Phase-Field-DNN model is higher than that of Phase-Field model and DNN model through the analysis of the classification results of binary classification and multi-classification problems with this data. In addition, the model we proposed is used to classify the normal and abnormal brain MRIs, and the classification results are compared with those of others. After comparison, we find that our proposed model achieve the best classification results. [ABSTRACT FROM AUTHOR]

• Introducing a novel nearest neighbourhood rule for kNN ensemble. • Improving classification performance in data with diverse patterns. • Building a kNN ensemble to achieve high classification performance by making the base models diversified. • Availability of the proposed method as an R package implementation on CRAN (https://cran.r-project.org/web/packages/ExNRuleEnsemble/index.html). k NN based ensemble methods minimise the effect of outliers by identifying a set of data points in the given feature space that are nearest to an unseen observation in order to predict its response by using majority voting. The ordinary ensembles based on k NN find out the k nearest observations in a region (bounded by a sphere) based on a predefined value of k. This scenario, however, might not work in situations where the test observation follows the pattern of the closest data points with the same class that lie on a certain path not contained in the given sphere. This paper proposes a k nearest neighbour ensemble where the neighbours are determined in k steps. Starting from the first nearest observation of the test point, the algorithm identifies a single observation that is closest to the observation at the previous step. At each base learner in the ensemble, this search is extended to k steps on a random bootstrap sample with a random subset of features selected from the feature space. The final predicted class of the test point is determined by using a majority vote in the predicted classes given by all base models. This new ensemble method is applied on 20 benchmark datasets and compared with other classical methods, including k NN based models, in terms of classification accuracy, kappa and Brier score as performance metrics. Boxplots are also utilised to illustrate the difference in the results given by the proposed and other state-of-the-art methods. The proposed method outperformed the considered classical methods in the majority of cases. The proposed method is further assessed through a detailed simulation study. [ABSTRACT FROM AUTHOR]

• Review of the most commonly used distance measures in machine learning • Analysis of their main properties, applications and key aspects to consider • The similarity analysis shows a high degree of correlation between all the measures • Evaluation of classification and clustering performance, noise tolerance and runtime • Canberra distance shows the best overall performance and the highest tolerance to noise Many machine learning and data mining tasks are based on distance measures, so a large amount of literature addresses this aspect somehow. Due to the broad scope of the topic, this paper aims to provide an overview of the use of these measures in the most common machine learning problems, pointing out those aspects to consider to choose the most appropriate measure for a particular task. For this purpose, the most recent works addressing the subject were reviewed and seven of the most commonly used measures were analyzed, investigating in detail their main properties and applications. Different experiments were carried out to study their relationships and compare their performance. The degradation of the results in the presence of noise was also considered, as well as the execution time required by each measure. [ABSTRACT FROM AUTHOR]

• We propose a multilabel feature selection method that considers the trade-off between the relevance of the features and the feature cost. • We derive a score function specialized as a cost-constrained multilabel feature selection problem based on normalization. • We identify the bound of the cost factor between the relevance of the features and the importance of the feature cost. • We devise a new framework to handle the shortage of datasets containing feature costs. Feature selection is one of the key steps in building a predictive model in multi-label classification. However, most of the existing methods do not take into account information about the costs associated with considered features, such as the costs of performing diagnostic medical tests. We consider a problem of cost-constrained multilabel feature selection, which aims to select a feature subset relevant to multiple labels while satisfying a user-specific maximal admissible budget. This approach allows for building a model with high predictive power, for which the cost of making a prediction for a single instance does not exceed the user-specified budget. In this problem, the balance between the feature subset relevance and its cost should be considered concurrently, which is nontrivial in practice because their optimal balance is unknown. In this paper, we propose a novel criterion for cost-constrained multilabel feature selection that combines the relevance and cost of the candidate feature. The relevance measure is derived using the lower bound of the mutual information between the feature subset and label vector. Moreover, we propose an effective method for determining the cost-factor value that controls the trade-off between relevancy and cost. The experimental results on multilabel datasets with various characteristics demonstrate the superiority of the proposed method over conventional methods. [ABSTRACT FROM AUTHOR]

• We propose a dynamic stroke classification approach, named DyGAT, for online handwritten documents and sketches. Different from previous methods, it can make real-time prediction during writers' writing process. • DyGAT is featured by a novel graph construction algorithm, which enables graph neural networks to better exploit the contextual information under the time constraint. • To evaluate the effectiveness and generality of DyGAT, we apply it to sketch semantic segmentation, document layout analysis, and diagram recognition. Extensive experiments have been conducted to analyze our method and demonstrate its superiority. Online handwriting is widely used in human-machine interface, education, office automation, and so on. Stroke classification for online handwritten documents and sketches aims to divide strokes into several semantic categories and is a necessary step for document recognition and understanding. Previous methods are essentially static in that they have to wait for the user to finish the whole sketch before making prediction. However, in practice, the more user-friendly way is to make real-time prediction as the user is writing. In this paper, we introduce Dynamic Graph ATtention network (DyGAT) to solve the dynamic stroke classification problem. The core of our method is to formalize a document/sketch into a multi-feature graph, in which nodes represent strokes, edges represent the relationships between strokes, and multiple nodes are applied to one stroke to control the information flow. The proposed method is general and is applicable to online handwritten data of many types. We conduct experiments on popular public datasets to perform sketch semantic segmentation, document layout analysis and diagram recognition, and experimental results show competitive performance. Particularly, the proposed method achieves stroke classification accuracies which are only slightly lower than those of static classification. [ABSTRACT FROM AUTHOR]

• Proposed an effective fine-grained visual classification method based on vision transformer. • Presented a core attention aggregator to mitigate the common information loss problem existing in the transformer layer. • Designed a new key token auto-selector based on information entropy to grasp the core discriminative features. • Further extending the application of vision transformer to fine-grained visual classification. The task of fine-grained visual classification (FGVC) is to distinguish targets from subordinate classifications. Since fine-grained images have the inherent characteristic of large inter-class variances and small intra-class variances, it is considered an extremely difficult task. Most existing approaches adopt CNN-based networks as feature extractors, which causes the extracted discriminative regions to contain most parts of the object in this way, thus failing to locate the really important parts. Recently, the vision transformer (ViT) has demonstrated its power on a wide range of image tasks, which uses an attention mechanism to capture global contextual information to establish a remote dependency on the target and thus extract more powerful features. Nevertheless, the ViT model still focuses more on global coarse-grained information rather than local fine-grained information, which may lead to its undesirable performance in fine-grained image classification. To this end, we redesigned an attention aggregating transformer (AA-Trans) to better capture minor differences among images by improving the ViT structure in this paper. In detail, we propose a core attention aggregator (CAA), which enables better information sharing between each transformer layer. Besides, we further propose an innovative information entropy selector (IES) to guide the network in acquiring discriminative parts of the image precisely. Extensive experiments show that our proposed model structure can achieve a new state-of-the-art performance on several mainstream datasets. [ABSTRACT FROM AUTHOR]

• In view of the necessity and research value, we are the first to specialize in studying fine-grained classification under the 3D point clouds representation, providing a new perspective for 3D shape classification. • Through in-depth analysis of the characteristics of the target object (3D point clouds) and the key to fine-grained classification tasks, design a feature extraction model to effectively learn the discriminative features. • To highlight discriminative local regions and captures spatial differences between 3D point clouds from different sub-categories, propose a module to capture spatial structure feature and aggerate local features. 3D point clouds classification has been a hot research topic and received great progress in recent years. However, due to the similar data distributions and subtle differences among various sub-categories in a meta-category, the 3D point clouds classification at a fine-grained level is still very challenging, especially without the annotations of part locations or attributes. In this paper, we propose a novel weakly supervised network for fine-grained 3D point clouds classification, namely FGPNet. Different from the previous supervised fine-grained classification methods that use class labels and other manual annotation information, FGPNet develops a unified framework to address both local geometric details and global spatial structures only using the class labels as input. Specifically, FGPNet firstly employs a context-aware discriminative feature extraction (CDFE) module, which extract contextual contrasted information across differential receptive fields hierarchically, and further capture discriminative local details from point clouds. Subsequently, an SimAM-Capsule Aggregation (SCA) module is introduced to highlight the significant local features and capture their spatial relationships. Quantitative and qualitative experimental results on fine-grained dataset including three categories Airplane, Chair and Car demonstrate that FGPNet outperforms the state-of-the-art methods on fine-grained 3D point clouds classification tasks. [ABSTRACT FROM AUTHOR]

• The concept of capacitive empirical risk function (CERF) is introduced, by using notions of capacity and Choquet integral. • CERF provides joint reduction in empirical risk function and confidence interval of a classification process, enhancing thereby the generalization ability of developed models. • Furthermore, CERF is demonstrated to capture interdependence of training samples, adding further improvement to the capacitive modeling process. • CERF-based capacitive pattern classification and capacitive bag-of-words (Cap BoW) models are developed by using the CERF as a cost function for the classification stage. • Tangible modeling enhancement is observed when using the CERF in constructing both capacitive pattern classification and the Cap BoW, compared with available empirical risk function-based techniques. This paper proposes capacitive bag-of-words (Cap BoW) modeling and capacitive pattern classification processes for improved generalization ability. The Cap BoW and capacitive pattern classification are realized by introducing the notion of the capacitive empirical risk function (CERF). The CERF is used as a cost function to build a capacitive pattern classification process. The resulted capacitive pattern classification is used in the classification stage of the bag-of-words process, realizing thereby the Cap BoW process. The use of the CERF in building the capacitive pattern classification and Cap BoW processes is demonstrated to achieve simultaneous reduction to the empirical risk function (ERF) and confidence interval, reducing potential overfitting and enhancing the generalization ability of considered models. To have a thorough evaluation, two groups of experiments are carried out. The first group addresses the capacitive pattern classification process for 4 datasets. The second group addresses a more holistic impact of the CERF by using Cap Bow model to build capacitive dual ergodicity limits-based bag-of-words (Cap DEL-BoW) process, which is applied to 5 image datasets. In both groups of experiments, remarkable enhancement is demonstrated with the use of CERF-based pattern classification and Cap BoW processes. Comparison with corresponding conventional non-capacitive models demonstrates the tangible enhancement with the use of CERF-based models. [ABSTRACT FROM AUTHOR]

• We prove theoretically and exemplify that information-theoretic metrics, e.g., MI or CMI, are just the mathematical expectations of corresponding probability-theoretic metrics, and the non-negativity of information-theoretic metrics doesn't hold while dealing with specific instances. The probability-theoretic metrics are introduced to verify the independence assumptions and then applied to reduce structure complexity and computational complexity. • To alleviate the i.i.d assumption and achieve global optimum, the pointwise log likelihood function P L L (B | t) is introduced to measure the number of bits encoded in the network topology of B learned from specific instance t, and then assigned to SPODE members in AODE as self-adaptive weighting metric. This learning approach can help approximate the true joint probability and improve the generalization performance. • We compare the performance of our classifier with other state-of-the-art classifiers on 36 datasets, ranging in size from 57 to 1,025,010 instances and 3 to 64 attributes. We show that our algorithm achieves remarkable classification performance in terms of zero-one loss, RMSE, bias-variance decomposition and conditional log likelihood (CLL) function. Recent studies have shown that Bayesian network classifiers (BNCs) are powerful tools for knowledge representation and classification, and averaged one-dependence estimators (AODE) is one of the most popular and effective BNCs since it can achieve the tradeoff between bias and variance due to its independence assumptions and ensemble learning strategy. However, unverified independence assumptions may result in biased estimates of probability distribution and then degradation in classification performance. In this paper, we prove theoretically the uncertainty of probability-theoretic independence and propose to measure the independence between attribute values implicated in specific instance. The estimates of conditional probability can be finely tuned based on point-wise independence analysis. Point-wise log likelihood function is then applied as weighting metric for committee members of AODE to improve the estimate of joint probability. Extensive experiments on 36 benchmark datasets show that, compared to other state-of-the-art classifiers, weighted one-dependence estimators using point-wise independence analysis can achieve competitive classification performance in terms of zero-one loss, RMSE, bias-variance decomposition and conditional log likelihood. [ABSTRACT FROM AUTHOR]

• We explore the commutative relationship between support and query samples and design a novel cycle optimization metric network for few-shot classification tasks. • We construct a cycle consistency loss in the mutual prediction process of support and query samples. The Forward network provides the prediction from support set to query set, then the Reverse network conducts the backward prediction. The Forward-Reverse network framework can achieve better performance under the cycle consistency loss. • We also introduce an optimization module, which can help correct the predicted results via an optimization loss to further improve the performance of our model. Our method outperforms existing few-shot learning methods on various datasets. Metric learning methods are widely used in few-shot learning due to their simplicity and effectiveness. Most existing methods directly predict query labels by comparing the similarity between support and query samples. In this paper, we design a cycle optimization metric network for few-shot classification task that optimizes model performance based on loop-prediction of the labels of query samples and support samples. Specifically, we construct a forward network and reverse network based on a geometric algebra Graph Neural Network (GA-GNN). These two networks form the loop prediction from support samples to query samples and then back to support samples, guided by a cycle-consistency loss. We also introduce an optimization module that is able to correct the predicted results of query samples to further improve the network performance. Our extensive experimental results demonstrate that the proposed cycle optimization metric network outperforms existing state-of-the-art few-shot learning methods on classification tasks. [ABSTRACT FROM AUTHOR]

• Design vision-language multimodal attention-based model VLCDoC for document analysis. • InterMCA and IntraMSA attention modules effectively align the crossmodal features. • Multimodal contrastive pretraining is proposed to learn vision-language features. • A good generality of the learned multimodal domain-agnostic features is demonstrated. Multimodal learning from document data has achieved great success lately as it allows to pre-train semantically meaningful features as a prior into a learnable downstream task. In this paper, we approach the document classification problem by learning cross-modal representations through language and vision cues, considering intra- and inter-modality relationships. Instead of merging features from different modalities into a joint representation space, the proposed method exploits high-level interactions and learns relevant semantic information from effective attention flows within and across modalities. The proposed learning objective is devised between intra- and inter-modality alignment tasks, where the similarity distribution per task is computed by contracting positive sample pairs while simultaneously contrasting negative ones in the joint representation space. Extensive experiments on public benchmark datasets demonstrate the effectiveness and the generality of our model both on low-scale and large-scale datasets. [ABSTRACT FROM AUTHOR]

• It is found that the classification ambiguity exists in both the inference and training processes to different degrees due to the non-correlation between classification and localization. • The proposed localization-classification sequence detector adopts the refined anchors as the intermediates to bridge the gap between the classification and localization, thereby mitigating the classification ambiguity. • The refinement-aware classification branch and assignment are proposed to perform refinement-aware classification and overcome corresponding obstacles. • The proposed method can mitigate the classification ambiguity to a large extent and achieve stable improvement across different baselines. In anchor-based detectors, the confidence scores and label-assignment results for the classification task are determined by the unrefined anchors rather than the final-refined boxes, which causes classification ambiguity due to the lack of correlation between the classification and localization tasks. In this paper, we investigate the classification ambiguity thoroughly via extensive experiments, and present the localization-classification sequence detector (LCSDet) that performs localization and classification in order, bridging the gap between them. To achieve this, the refinement-aware (RA) classification branch and RA assignment are proposed in LCSDet. In inference, the RA classification branch rectifies the feature misalignment and directly classifies the refined anchors. During training, the RA assignment tackles the training instability, narrows the location-quality gap and assigns the refined anchors to ground-truth objects. Comprehensive experiments indicate that the LCSDet can effectively mitigate the classification ambiguity and achieve stable improvement across different baselines. [ABSTRACT FROM AUTHOR]

• LASA fills the gap that no adversary is designed specified for sonar image, according to the real-world underwater conditions. • The performance of LASA is superior to the existing state-of-the-art adversarial attack methods (both targeted and non-targeted) in SSS image classification. • We demonstrate that the adversarial training indeed improves the robustness of CNNs against several adversarial attack methods. • LASA shows better understanding of adversarial attack on SSS images. Deep convolutional neural networks (CNNs) are extensively applied to the classification tasks for Side-scan sonar (SSS) images. However, state-of-the-art neural networks are prone to be confused by adversarial attacks that generate a tiny modification of the images, threatening the security of SSS classification. The robustness of CNN to adversarial attacks can be improved by introducing adversarial examples through adversarial training. Practical adversarial examples are often generated from elaborate adversarial attackers. For the underwater scenario of sonar, a specially designed adversarial attack method to weaken SSS image classification can make the research community better understand the weakness of CNN in this scenario and improve the security measures in a well-directed way. Thus, exploring adversarial attack methods for SSS image classification is essential. Nevertheless, the existing adversarial attack methods are designed for optical images, reflecting no physical characteristics of sonar images. To fill this gap and investigate the adversarial attack related to real-world conditions, in this paper, we propose an adversarial attack method named Lambertian Adversarial Sonar Attack (LASA). It initially leverages the Lambertian model to simulate the formation of the SSS image, factoring the image to three parameters, then updates the parameters on the direction of gradients by the chain rule. Finally, the parameters regenerate the adversarial example to fool the classifier. To validate the performance of LASA, we constructed a diversified SSS image dataset containing three categories. On our dataset, LASA reduces the Top-1 accuracy of a well-trained ResNet-101 to 7.31 % ± 0.21 (one-shot version) and 0.00% (iterative version), the success rate of targeted attack reaches 97.03 ± 2.24 , far beyond the performance of the existing state-of-the-art adversarial attack methods. Meanwhile, we show that the adversarial training using examples generated from LASA makes the classifier more robust. We expect that our methods can be applied as a benchmark of adversarial attacks on SSS images, motivating future research to design novel neural networks or defensive methods to resist real-world adversarial attacks on SSS images. [ABSTRACT FROM AUTHOR]

• We formalize a new learning framework named Multi-Dimensional Multi-Label (MDML) classification to learn from objects whose semantics need to be characterized with multiple heterogeneous label spaces as well as multi-label annotations in each label space. • We specifically design an approach named CLIM to solve the MDML problem which can consider label correlations within individual dimension and across multiple dimensions. • We evaluate the performance of CLIM over four real-world MDML data sets and the experimental results clearly validate the effectiveness of the proposed approach. In traditional classification framework, the semantics of each object is usually characterized by annotating a single class label from one homogeneous label space. Nonetheless, objects with rich semantics naturally arise in real-world applications whose properties need to be characterized in a more sophisticated manner. In this paper, a new classification framework named Multi-Dimensional Multi-Label (MDML) classification is investigated which models objects with rich semantics by encompassing heterogeneous label spaces and multi-label annotations. Specifically, MDML generalizes the traditional classification framework by assuming a number of heterogeneous label spaces to characterize semantics from different dimensions, where each object is further annotated with multiple class labels from each heterogeneous label space. To learn from MDML training examples, a first attempt named CLIM is proposed based on an augmented stacking strategy. Firstly, CLIM induces a base multi-label predictive model w.r.t. each label space by maximizing the likelihood of the observed multiple class labels. Secondly, the thresholding predictions from all base models are used to augment the original feature space to yield stacked multi-label predictive models. The two-level models are refined alternately via empirical threshold tuning. Experiments on four real-world MDML data sets validate the effectiveness of CLIM in learning from training examples with heterogeneous label spaces and multi-label annotations. [ABSTRACT FROM AUTHOR]

• We propose a granularity-aware distillation module to enhance the representation ability of the model. We adopt a multi-granularity feature fusion learning strategy to jointly learn multi-level information, and use cross-layer self-distillation regularization to improve the robustness of features at different granularity levels. • We propose a structure modeling region proposal module. Based on the collaborative learning of discriminative semantics and structural semantics under weak supervision, the model can improve the ability to capture multi-granularity discriminative regions and mine regional structural semantic associations. • GDSMP-Net reports state-of-the-art performance on four widely-used challenging datasets, including CUB-200-2011, Stanford Cars, FGVC-Aircraft and NA-birds. Fine-grained visual classification (FGVC) aims to identify objects belonging to multiple sub-categories of the same super-category. The key to solving fine-grained classification problems is to learn discriminative visual feature representation with only subtle differences. Although previous work based on refined feature learning has made great progress, however, high-level semantic features often lack key information for fine-grained visual object nuances. How to efficiently integrate semantic information of different granularities from classification networks is a critical. In this paper, we propose Granularity-aware Distillation and Structure Modeling region Proposal Network(GDSMP-Net). Our solution integrates multi-granularity hierarchical information through a multi-granularity fusion learning strategy to enhance feature representation. In view of the inherent challenges of large intra-class differences in FGVC, a cross-layer self-distillation regularization is proposed to to strengthen the connection between high-level semantics and low-level semantics for robust multi-granularity feature learning. On this basis, we use a weakly supervised method to generate local branches, and the collaborative learning of discriminative semantics and structural semantics based on local regions, facilitating model to perceive contextual information to capture structural interactions between local semantics. Comprehensive experiments show that our method achieves state-of-the-art performance on four widely-used challenging datasets.(CUB-200-2011, Stanford Cars, FGVC-Aircraft and NA-birds). [ABSTRACT FROM AUTHOR]

• A novel framework named graph auxiliary augmentation learning (GAU) is proposed, which co-trains the primary task together with a fine-grained auxiliary classification through a multi-task GNN. • It alleviates the sensitivity of the model to the pseudo-label quality and reduces the model degradation due to the accumulative error of the pseudo-labels. • The fine-grained auxiliary classification task helps to learn better node representations from a different view, thereby boosting the performance of the primary task. • It is architecture-agnostic so that it can be applied to any variant of GNN. Experiments show that it can achieve superior performance on different architectures when compared with other state-of-the-art methods. Node classification has become an important research topic in recent years. Since there are always a few training samples, researchers improve the performance by properly leveraging the predictions of unlabeled nodes during training. However, suffering from the model degradation resulted from the accumulative error of pseudo-labels, there is limited improvement. In this paper we present fine-grained G raph A uxiliary a U gmentation (GAU). It trains the primary task together with an automatically created auxiliary task which is a fine-grained node classification task. And an auxiliary augmentation strategy is designed to enlarge the labeled set for the auxiliary task by utilizing the pseudo-labels of the primary task. Comprehensive experiments show that GAU alleviates the sensitivity of the model to the pseudo-label quality, so more unlabeled nodes can participate in the training. From the perspective of co-training, the fine-grained auxiliary task which is trained by much more unlabeled nodes helps to learn better node representations from a different view, thereby boosting the final performance. Extensive experiments verify the superior performance of the GAU on different GNN architectures when compared with other state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

• The paper proposes a high-level semantic mapping within semantic space to increase the semantic deduction ability of the deep neural network. • Different from multi-level label learning, the proposed model can not only learn features but also deduct high-level semantic expression by itself. • The deduction learning is realized by the semantic prior and the proposed random search for opposite semantic to ensure the smoothness of semantic clustering. • The proposed model achieves stable convergence and high classification accuracy. It can be taken as a plug-in module for various deep learning applications. The paper proposes a semantic clustering based deduction learning by mimicking the learning and thinking process of human brains. Human beings can make judgments based on experience and cognition, and as a result, no one would recognize an unknown animal as a car. Inspired by this observation, we propose to train deep learning models using the clustering prior that can guide the models to learn with the ability of semantic deducing and summarizing from classification attributes, such as a cat belonging to animals while a car pertaining to vehicles. The proposed approach realizes the high-level clustering in the semantic space, enabling the model to deduce the relations among various classes during the learning process. In addition, the paper introduces a semantic prior based random search for the opposite labels to ensure the smooth distribution of the clustering and the robustness of the classifiers. The proposed approach is supported theoretically and empirically through extensive experiments. We compare the performance across state-of-the-art classifiers on popular benchmarks, and the generalization ability is verified by adding noisy labeling to the datasets. Experimental results demonstrate the superiority of the proposed approach. [ABSTRACT FROM AUTHOR]

• Conditional probability twin SVM (CPTWSVM) is presented to estimate the conditional probability function and the empirical risk of each class. • CPTWSVM not only can measure the misclassification of the training samples for each class, but also returns the discriminant projections and distribution function of each class. • CPTWSVM can eliminate the problems of inconsistent measurement in TWSVMs. • CPTWSVM can be extended to multiclass classification and maintain the above properties. • Experiments show that CPTWSVM has a better performance than some leading SVMs and TWSVMs communities. In this paper, we estimate the conditional probability function by presenting a new twin SVM model (CPTWSVM) in binary and multiclass classification problems. The motivation of CPTWSVM is to implement the empirical risk minimization on training data, which is hard to realize in traditional twin SVMs. In each subproblem of CPTWSVM, it measures the empirical risk and outputs the corresponding probability estimate of each class, which eliminates the problems of inconsistent measurement in twin SVMs. Though an additional discriminant objective function is introduced, the optimization problem size of each subproblem is smaller than conditional probability SVM, and is solved by block decomposition algorithm efficiently. In addition, we extend CPTWSVM to multiclass classification by estimating the conditional probability of each class, and maintaining the above properties. Numerical experiments on benchmark and real application datasets demonstrate that CPTWSVM outputs the estimate of probability and the data projection well, resulting in better generalization ability than some leading TWSVMs communities, in terms of binary and multiclass classification. [ABSTRACT FROM AUTHOR]

• Finite training sets lead to classifier error rates above the Bayes error rate. • The training set size required in a practical setting is a relevant information. • Most approaches are purely experimental or consider just two classes. • A theoretical expression for the relative excess over the Bayes error rate is provided. In this paper, a theoretical learning curve is derived for the multi-class Bayes classifier. This curve fits general multivariate parametric models of the class-conditional probability density. The derivation uses a proxy approach based on analyzing the convergence of a statistic which is proportional to the posterior probability of the true class. By doing so, the curve depends only on the training set size and on the dimension of the feature vector; it does not depend on the model parameters. Essentially, the learning curve provides an estimate of the reduction in the excess of the probability of error that can be obtained by increasing the training set size. This makes it attractive in order to deal with the practical problems of defining appropriate training set sizes. [ABSTRACT FROM AUTHOR]

• A two-parallel-path super-network is carefully designed from three levels: operation-level, channel-level and training-level. • Instead of using two paths to train the super-network, we adopt the binary gate to randomly eliminate at most one path to reduce the memory cost. • We adopt the partially-connected strategy to further reduce the memory cost, where each sub-network only adopts partial channels. • Sigmoid function is introduced to select the best input for each node across two operator sub-sets. • The proposed method can obtain better results in speed and effect. Neural Architecture Search (NAS) has made remarkable progress in automatic machine learning. However, it still suffers massive computing overheads limiting its wide applications. In this paper, we present an efficient search method, Fast Parallel Differential Neural Architecture Search (FP-DARTS). The proposed method is carefully designed from three levels to construct and train the super-network. Firstly, at the operation-level, to reduce the computational burden, different from the standard DARTS search space (8 operations), we decompose the operation set into two non-overlapping operator sub-sets (4 operations for each). Adopting these two reduced search spaces, two over-parameterized sub-networks are constructed. Secondly, at the channel-level, the partially-connected strategy is adopted, where each sub-network only adopts partial channels. Then these two sub-networks construct a two-parallel-path super-network by addition. Thirdly, at the training-level, the binary gate is introduced to control whether a path participates in the super-network training. It may suffer an unfair issue when using softmax to select the best input for intermediate nodes across two operator sub-sets. To tackle this problem, the sigmoid function is introduced, which measures the performance of operations without compression. Extensive experiments demonstrate the effectiveness of the proposed algorithm. Specifically, FP-DARTS achieves 2.50% test error with only 0.08 GPU-days on CIFAR10, and a state-of-the-art top-1 error rate of 23.7% on ImageNet using only 2.44 GPU-days for search. [ABSTRACT FROM AUTHOR]

• A novel framework of joint optimization of scoring and thresholding models is proposed for online multi-label classification. • Based on this framework, two types of algorithms are proposed, one of which adaptively learns an instance-based thresholding model and the other of which uses a fixed thresholding model. • For each type of algorithms, a first-order method and a second-order one are provided for updating the online multi-label classifier. • All the proposed algorithms can be proved to achieve a sub-linear regret. Using Mercer kernels, two first-order algorithms can be easily extended to handle nonlinear multi-label prediction tasks. • Experiments show the advantage of the proposed algorithms, in terms of various multi-label performance metrics. Existing online multi-label classification works cannot well handle the online label thresholding problem and lack regret analysis for their online algorithms. This paper proposes a novel framework of joint optimization of scoring and thresholding models for online multi-label classification, with the aim to overcome the above drawbacks. The key feature of our framework is that both scoring and thresholding models are included as important components of the online multi-label classifier and are incorporated into one online optimization problem. Based on this framework, we present two adaptive label thresholding algorithms and two fixed thresholding algorithms. For each type of algorithms, a first-order method and a second-order one are provided for updating the online multi-label classifier. Both methods enjoy a closed-form update. Our proposed algorithms are proved to achieve a sub-linear regret. Using Mercer kernels, two first-order algorithms can be extended to handle nonlinear multi-label prediction tasks. Experiments show the advantage of the adaptive and the fixed thresholding algorithms, in terms of various multi-label performance metrics. [ABSTRACT FROM AUTHOR]

• We approach object point cloud classification from a more practical perspective, and propose the single-view, partial setting under which point clouds covering the partial surface of object instances are observed. • We discuss the limitations of existing methods, and show that their performance drops drastically under the practical setting. • We propose a baseline method of Pose-Accompanied Point cloud classification Network (PAPNet), which accompanies the classification task with an auxiliary one of supervised object pose learning. • To advance the research field, we adapt existing ModelNet40 and ScanNet benchmarks to the single-view, partial setting. Object point cloud classification has drawn great research attention since the release of benchmarking datasets, such as the ModelNet and the ShapeNet. These benchmarks assume point clouds covering complete surfaces of object instances, for which plenty of high-performing methods have been developed. However, their settings deviate from those often met in practice, where, due to (self-)occlusion, a point cloud covering partial surface of an object is captured from an arbitrary view. We show in this paper that performance of existing point cloud classifiers drops drastically under the considered single-view, partial setting; the phenomenon is consistent with the observation that semantic category of a partial object surface is less ambiguous only when its distribution on the whole surface is clearly specified. To this end, we argue for a single-view, partial setting where supervised learning of object pose estimation should be accompanied with classification. Technically, we propose a baseline method of Pose-Accompanied Point cloud classification Network (PAPNet) ; built upon S E (3) -equivariant convolutions, the PAPNet learns intermediate pose transformations for equivariant features defined on vector fields, which makes the subsequent classification easier (ideally) in the category-level, canonical pose. By adapting existing ModelNet40 and ScanNet datasets to the single-view, partial setting, experiment results can verify the necessity of object pose estimation and superiority of our PAPNet to existing classifiers. [ABSTRACT FROM AUTHOR]

Self-labeled techniques are semi-supervised classification models that overcome the shortage of labeled samples via an iterative process. Most relevant proposals are inspired by boosting schemes to iteratively enlarge labeled data, but these methods are constrained by the number and distribution of the initial labeled data. Up to the present, the only exceptions which can solve the above problem are SEG-SSC, k -means-SSC and LC-SSC. However, SEG-SSC relies on too many parameters. Besides, it is hard to improve the distribution of the initial labeled data when the initial labeled set can not roughly represent the distribution of the original data. k -means-SSC and LC-SSC fail to significantly improve the number of the initial labeled data by a limited number of representative points. To address the above issues, this paper proposes a framework based on local cores and synthetic examples generation for self-labeled semi-supervised classification (LCSEG-SSC). First, a new method for finding local cores on labeled and unlabeled data is proposed to improve the distribution of the initial labeled data. Second, STOPF or active labeling is used to predict found local cores. Third, a new example generation technique is proposed to create synthetic labeled samples, intending to improve the number of the initial labeled data. After that, any self-labeled with boosting schemes can be executed on the improved labeled data effectively. Intensive experiments prove that LCSEG-SSC outperforms state-of-the-art methods, especially in a relatively low ratio of labeled data. [ABSTRACT FROM AUTHOR]

COVID-19, as an infectious disease, has shocked the world and still threatens the lives of billions of people. Early detection of COVID-19 patients is an important issue for treating and controlling the disease from spreading. In this paper, a new strategy for detecting COVID-19 infected patients will be introduced, which is called Distance Biased Naïve Bayes (DBNB). The novelty of DBNB as a proposed classification strategy is concentrated in two contributions. The first is a new feature selection technique called Advanced Particle Swarm Optimization (APSO) which elects the most informative and significant features for diagnosing COVID-19 patients. APSO is a hybrid method based on both filter and wrapper methods to provide accurate and significant features for the next classification phase. The considered features are extracted from Laboratory findings for different cases of people, some of whom are COVID-19 infected while some are not. APSO consists of two sequential feature selection stages, namely; Initial Selection Stage (IS2) and Final Selection Stage (FS2). IS2 uses filter technique to quickly select the most important features for diagnosing COVID-19 patients while removing the redundant and ineffective ones. This behavior minimizes the computational cost in FS2, which is the next stage of APSO. FS2 uses Binary Particle Swarm Optimization (BPSO) as a wrapper method for accurate feature selection. The second contribution of this paper is a new classification model, which combines evidence from statistical and distance based classification models. The proposed classification technique avoids the problems of the traditional NB and consists of two modules; Weighted Naïve Bayes Module (WNBM) and Distance Reinforcement Module (DRM). The proposed DBNB tries to accurately detect infected patients with the minimum time penalty based on the most effective features selected by APSO. DBNB has been compared with recent COVID-19 diagnose strategies. Experimental results have shown that DBNB outperforms recent COVID-19 diagnose strategies as it introduce the maximum accuracy with the minimum time penalty. [ABSTRACT FROM AUTHOR]

The paper is devoted to two problems: (1) reinforcement of SVM algorithms, and (2) justification of memorization mechanisms for generalization. (1) Current SVM algorithm was designed for the case when the risk for the set of nonnegative slack variables is defined by l 1 norm. In this paper, along with that classical l 1 norm, we consider risks defined by l 2 norm and l ∞ norm. Using these norms, we formulate several modifications of the existing SVM algorithm and show that the resulting modified SVM algorithms can improve (sometimes significantly) the classification performance. (2) Generalization ability of existing learning algorithms is usually explained by arguments involving uniform convergence of empirical losses to the corresponding expected losses over a given set of functions. However, along with bounds for uniform convergence of empirical losses to the expected losses, the VC theory also provides bounds for relative uniform convergence. These bounds lead to a more accurate estimate of the expected loss. Advanced methods of estimating of expected risk of error have to leverage these bounds, which also support mechanisms of training data memorization, which, as the paper demonstrates, can improve classification performance. [ABSTRACT FROM AUTHOR]

• Prior works treat few-shot fine-grained classification and person search separately. • The proposed query-guided networks (QGN) address both tasks in a unified framework. • QGN introduces query-guidance for the task of few-shot fine-grained recognition. • State-of-the-art performance on CUB, FGVC-Aircraft, Stanford Dogs datasets. • The paper performs an in-depth analysis of each of the query-guided components of QGN. [Display omitted] Few-shot fine-grained classification and person search appear as distinct tasks and literature has treated them separately. But a closer look unveils important similarities: both tasks target categories that can only be discriminated by specific object details; and the relevant models should generalize to new categories, not seen during training. We propose a novel unified Query-Guided Network (QGN) applicable to both tasks. QGN consists of a Query-guided Siamese-Squeeze-and-Excitation subnetwork which re-weights both the query and gallery features across all network layers, a Query-guided Region Proposal subnetwork for query-specific localisation, and a Query-guided Similarity subnetwork for metric learning. QGN improves on a few recent few-shot fine-grained datasets, outperforming other techniques on CUB by a large margin. QGN also performs competitively on the person search CUHK-SYSU and PRW datasets, where we perform in-depth analysis. [ABSTRACT FROM AUTHOR]

• Transferring the training experience among constituent networks facilitates semi-supervised image classification. • We design a collaborative learning mechanism based on unreliability adaptation among constituent networks. • We improve the complementarity of constituent networks by resisting adversarial perturbation. • The unreliability adaptation and perturbation-based regularization lead to the superior performance on multiple datasets. Constructing training goals for unlabeled data is crucial for image classification in the semi-supervised setting. Consistency regularization typically encourages a model to produce consistent predictions with the given training goals, while unreliability adaptation aims to learn the transition probabilities from model predictions to training goals, instead of enforcing their consistency. In this paper, we present a model of Collaborative learning with Unreliability Adaptation (CoUA), in which multiple constituent networks collaboratively learn with each other by adapting their predictions. Toward this end, an additional adaptation module is incorporated into each network to learn a transition probability from its own prediction to that of the paired network. Therefore, the networks can exchange training experience, without being overly sensitive to the unreliability of predictions. To further enhance the collaborative learning, each network is encouraged to produce consistent predictions with the consensus results, while being resistant to the adversarial perturbations against others. Therefore, the networks are able to mutually reinforce each other. We perform extensive experiments on multiple image classification benchmarks to verify the superiority of the co-adaptation based collaborative learning mechanism. [ABSTRACT FROM AUTHOR]

• Propose three noise-robust mechanisms to address the noise generation problem in classic oversampling algorithms: adopting an advanced clustering algorithm, designing adaptive embedding to generate samples, and implementing a safe boundary to enlarge class boundaries. • Propose the heterogeneous distance metric to better cluster mixed-type data along with dedicated approaches to avoid generating groundless samples with categorical variables. • Adapted decomposition strategy extends solution for binary imbalanced data to the multi-class setting. Moreover, better placement of new samples are provided. • Experiments on the standard datasets validate the effectiveness of the proposed data. The class imbalance problem is characterized by an unequal data distribution in which majority classes have a greater number of data samples than minority classes. Oversampling methods generate samples for minority classes to balance the data distribution. However, the generated minority samples may overlap with majority samples, resulting in noise. In this paper, we propose a noise-robust oversampling algorithm for mixed-type and multi-class imbalanced data. Our proposed noise-robust designs include an algorithm to eliminate noise within clusters of data samples, adaptive embedding to generate samples safely, and a safe boundary for enlarging class boundaries. The heterogeneous distance metric and adapted decomposition strategy render our noise-robust algorithm suitable for mixed-type and multi-class imbalanced data. Experimental results on 20 benchmark datasets demonstrate the effectiveness of the proposed algorithm. [ABSTRACT FROM AUTHOR]

• Decouple the causality among skin lesion classification, detection and segmentation from the Pareto efficiency view, solving the common trade-off dilemma in multiple tasks. • New paradigm boosts the diagnosis with the investigated favorable fusing manner. • Effective model compression scheme overcomes the redundant complexity, achieving the raised performance under fewer parameters. Multi-task learning has been used widely in many computer aided diagnosis applications recently, while the trade-off between different tasks remains challenging. Also, the inherent causality is less studied. In this paper, we focus on skin lesion analysis, including lesion classification, detection and segmentation. By defining the chain relationship (i.e., lesion detection boosts contour segmentation, and segmentation boosts lesion classification in turn), and further decoupling each pair-wise causality (e.g., detection to segmentation) from the Pareto efficiency view, we can solve the common trade-off issue between multi-task. On this basis, we propose a novel paradigm to improve the skin lesion segmentation and classification separately, and favourable feature fusion ways for each task are explored. Moreover, to address the huge model size problem, we design an effective model compression scheme (MCS). Extensive experiments on the ISIC2017 and PH2 datasets are conducted to evaluate the proposed paradigm. The results demonstrate that the popular models such as ResNet, DenseNet and UNet for lesion analysis can be boosted by applying the proposed paradigm, and the designed MCS reduces the amount of model parameters efficiently. We achieve performance improvements on skin lesion segmentation and classification without strenuous network design and soaring model complexity. This proposed approach is promising for the multi-task diagnosis setting in other medical applications. [ABSTRACT FROM AUTHOR]

• We propose a novel multi-source domain adaptation method for classification. • We leverage domain consistency regularization for effective feature alignment. • An adaptive weighting strategy is designed to tackle negative transfer. • The proposed method achieves state-of-the-art performance on multiple benchmarks. Deep learning-based multi-source unsupervised domain adaptation (MUDA) has been actively studied in recent years. Compared with single-source unsupervised domain adaptation (SUDA), domain shift in MUDA exists not only between the source and target domains but also among multiple source domains. Most existing MUDA algorithms focus on extracting domain-invariant representations among all domains whereas the task-specific decision boundaries among classes are largely neglected. In this paper, we propose an end-to-end trainable network that exploits domain Consistency Regularization for unsupervised Multi-source domain Adaptive classification (CRMA). CRMA aligns not only the distributions of each pair of source and target domains but also that of all domains. For each pair of source and target domains, we employ an intra-domain consistency to regularize a pair of domain-specific classifiers to achieve intra-domain alignment. In addition, we design an inter-domain consistency that targets joint inter-domain alignment among all domains. To address different similarities between multiple source domains and the target domain, we design an authorization strategy that assigns different authorities to domain-specific classifiers adaptively for optimal pseudo label prediction and self-training. Extensive experiments show that CRMA tackles unsupervised domain adaptation effectively under a multi-source setup and achieves superior adaptation consistently across multiple MUDA datasets. [ABSTRACT FROM AUTHOR]

• A novel texture descriptor to address both scale transformation and noise interference. • An extended LBP with lightweight feature dimension. • Maintains both macro and micro descriptive information in the spatial and spectral domains. • Outperforms thirty classical LPB variants as well as eight typical deep learning methods. • Experiments on five public databases and one fresh texture database. As one of the most successful local feature descriptors, the local binary pattern (LBP) estimates the texture distribution rule of an image based on the signs of differences between neighboring pixels to obtain intensity- and rotation- invariance. In this paper, we propose a novel image descriptor to address scale transformation and noise interference simultaneously. We name it scale-selective and noise-robust extended LBP (SNELBP). First, each image in training sets is transformed into different scale spaces by a Gaussian filter. Second, noise-robust pattern histograms are obtained from each scale space by using our previously proposed median robust extended LBP (MRELBP). Then, scale-invariant histograms are determined by selecting the maximum among all scale levels for a certain image. Finally, the most informative patterns are selected from the dictionary pretrained by the two-stage compact dominant feature selection method (CDFS), maintaining the descriptor more lightweight with sufficiently low time cost. Extensive experiments on five public databases (Outex_TC_00011, TC_00012, KTH-TIPS, UMD and NEU) and one fresh texture database (JoJo) under two kinds of interferences (Gaussian and salt pepper) indicate that our SNELBP yields more competitive results than thirty classical LPB variants as well as eight typical deep learning methods. [ABSTRACT FROM AUTHOR]

• Three variants of Ensemble Deep Random Vector Functional Link network are proposed. • Batch normalization is introduced to the edRVFL network to re-normalize the hidden features. • Weighting and pruning methods are employed to improve the classification ability. • No re-training of previous layers is required for these networks when adding one more hidden layer. • Our best method outperforms other 13 classifiers on 24 datasets. In this paper, we first integrate normalization to the Ensemble Deep Random Vector Functional Link network (edRVFL). This re-normalization step can help the network avoid divergence of the hidden features. Then, we propose novel variants of the edRVFL network. Weighted edRVFL (WedRVFL) uses weighting methods to give training samples different weights in different layers according to how the samples were classified confidently in the previous layer thereby increasing the ensemble's diversity and accuracy. Furthermore, a pruning-based edRVFL (PedRVFL) has also been proposed. We prune some inferior neurons based on their importance for classification before generating the next hidden layer. Through this method, we ensure that the randomly generated inferior features will not propagate to deeper layers. Subsequently, the combination of weighting and pruning, called Weighting and Pruning based Ensemble Deep Random Vector Functional Link Network (WPedRVFL), is proposed. We compare their performances with other state-of-the-art classification methods on 24 tabular UCI classification datasets. The experimental results illustrate the superior performance of our proposed methods. [ABSTRACT FROM AUTHOR]