Search

Knowledge distillation is a simple yet effective technique for deep model compression, which aims to transfer the knowledge learned by a large teacher model to a small student model. To mimic how the teacher teaches the student, existing knowledge distillation methods mainly adapt an unidirectional knowledge transfer, where the knowledge extracted from different intermedicate layers of the teacher model is used to guide the student model. However, it turns out that the students can learn more effectively through multi-stage learning with a self-reflection in the real-world education scenario, which is nevertheless ignored by current knowledge distillation methods. Inspired by this, we devise a new knowledge distillation framework entitled multi-target knowledge distillation via student self-reflection or MTKD-SSR, which can not only enhance the teacher's ability in unfolding the knowledge to be distilled, but also improve the student's capacity of digesting the knowledge. Specifically, the proposed framework consists of three target knowledge distillation mechanisms: a stage-wise channel distillation (SCD), a stage-wise response distillation (SRD), and a cross-stage review distillation (CRD), where SCD and SRD transfer feature-based knowledge (i.e., channel features) and response-based knowledge (i.e., logits) at different stages, respectively; and CRD encourages the student model to conduct self-reflective learning after each stage by a self-distillation of the response-based knowledge. Experimental results on five popular visual recognition datasets, CIFAR-100, Market-1501, CUB200-2011, ImageNet, and Pascal VOC, demonstrate that the proposed framework significantly outperforms recent state-of-the-art knowledge distillation methods. [ABSTRACT FROM AUTHOR]

The feature representation learning is the basic task that plays an important role in artificial intelligence, data mining and robotics. In [[17]], due to the ambiguity of the category boundary, the authors proposed Dempster-Shafer evidence theory (DST) to reduce category boundary ambiguity and output reasonable decisions by combining adaptation outputs based on uncertainty. Preface to the Special Issue "Advancement of Mathematical Methods in Feature Representation Learning for Artificial Intelligence, Data Mining and Robotics" - Special Issue Book Inspired by generative adversarial networks (GANs), the authors of [[18]] proposed a novel adversarial domain adaptation method with an initial state fusion strategy followed by a domain similarity strategy based on information entropy. [Extracted from the article]

Learning graph embeddings for high-dimensional data is an important technology for dimensionality reduction. The learning process is expected to preserve the discriminative and geometric information of high-dimensional data in a new low-dimensional subspace via either manual or automatic graph construction. Although both manual and automatic graph constructions can capture the geometry and discrimination of data to a certain degree, they working alone cannot fully explore the underlying data structure. To learn and preserve more discriminative and geometric information of the high-dimensional data in the low-dimensional subspace as much as possible, we develop a novel D iscriminative and G eometry- P reserving A daptive G raph E mbedding (DGPAGE). It systematically integrates manual and adaptive graph constructions in one unified graph embedding framework, which is able to effectively inject the essential information of data involved in predefined graphs into the learning of an adaptive graph, in order to achieve both adaptability and specificity of data. Learning the adaptive graph jointly with the optimized projections, DGPAGE can generate an embedded subspace that has better pattern discrimination for image classification. Results derived from extensive experiments on image data sets have shown that DGPAGE outperforms the state-of-the-art graph-based dimensionality reduction methods. The ablation studies show that it is beneficial to have an integrated framework, like DGPAGE, that brings together the advantages of manual/adaptive graph construction. • Predefined similarity and diversity adjacency graphs. • Inject the manual graphs into learning automatic graph. • Design DGPAGE via unifying the learning of projection and adaptive graph. [ABSTRACT FROM AUTHOR]

Collaborative representation-based classification (CRC), as a typical kind of linear representation-based classification, has attracted more attention due to the effective and efficient pattern classification performance. However, the existing class-specific representations are not competitively learned from collaborative representation for achieving more informative pattern discrimination among all the classes. With the purpose of enhancing the power of competitive and discriminant representations among all the classes for favorable classification, we propose a novel CRC method called the class-specific mean vector-based weighted competitive and collaborative representation (CMWCCR). The CMWCCR mainly contains three discriminative constraints including the competitive, mean vector and weighted constraints that fully employ the discrimination information in different ways. In the competitive constraint, the representations from any one class and the other classes are adapted for learning competitive representations among all the classes. In the newly designed mean vector constraint, the mean vectors of all the class-specific training samples with the corresponding class-specific representations are taken into account to further enhance the competitive representations. In the devised weighted constraint, the class-specific weights are constrained on the representation coefficients to make the similar classes have more representation contributions to strengthening the discrimination among all the class-specific representations. Thus, these three constraints in the unified CMWCCR model can complement each other for competitively learning the discriminative class-specific representations. To verify the CMWCCR classification performance, the extensive experiments are conducted on twenty-eight data sets in comparisons with the state-of-the-art representation-based classification methods. The experimental results show that the proposed CMWCCR is an effective and robust CRC method with satisfactory performance. • Design mean vector constraint for representation-based classification. • Design weighted constraint on the representation coefficients. • Propose CMWCCR method via competitive representation. [ABSTRACT FROM AUTHOR]

Intraspecific variation is a common phenomenon in nature, but to date, research on such variation in hematological traits of urodeles remains scarce. To compare intraspecific variation in hematological traits among populations, and explore whether snout–vent length (SVL) and body mass influence hematological traits, hematological parameters and erythrocyte size in 58 mature Batrachuperus tibetanus specimens belonging to three populations (Xihe, Meixian and Taibai) in northwestern China were analyzed. There were no sexual differences in any hematological trait for all populations. No hematological traits differed significantly between the Meixian and Taibai populations, but significantly lower values of erythrocyte count (RBC), hemoglobin (Hb), hematocrit (Hct), erythrocyte length (L), and erythrocyte area (A), as well as a significantly higher leukocyte count (WBC) were observed in the Xihe population compared with the other two. Linear regression analyses showed that significant relationships were present between SVL and Hb, RBC, Hct, and L; and body mass and Hb, RBC, Hct, and L. However, SVL and WBC were negatively correlated. Only L differed significantly among populations when accounting for the effects of SVL and body mass. Hb, RBC, WBC, and Hct variations among populations possibly contribute to differences in SVL or body mass, and erythrocyte size (L and A) variations are perhaps attributable to differences in lower Hb, RBC, and Hct; these variations are not the result of differences in habitat variables. Our results provide a foundation for understanding physiological intraspecific variation. [ABSTRACT FROM AUTHOR]

Knowledge distillation is a promising model compression technique that generally distills the knowledge from a complex teacher model to a lightweight student model. However, the performance gain of a student model is usually limited by the capacity gap between the large teacher model and the small student model. In this paper, we propose a new collaborative knowledge distillation method that makes use of a new strategy, named Filter Knowledge Transfer (FKT) to detect and learn the valuable filter information following from the teacher to the student. To be specific, the useful knowledge of filters measured by using information entropy is transferred between different peer networks and unimportant filters are reactivated according to the ratio based on the characterized filter information entropy during the online distillation process. Experimental results derived on four popular datasets, CIFAR-10/100, Market-1501, and Tiny-ImageNet, demonstrate the superiority of our proposed method over the others we considered. • Design filter knowledge transfer. • Propose a new collaborative knowledge distillation method. • Design filter knowledge entropy to reflect the importance of filter knowledge. [ABSTRACT FROM AUTHOR]

With recent success of deep neural networks, stacked autoencoder networks have received a lot of attention for robust unsupervised representation learning. However, recent autoencoder methods cannot make full use of multi-view information and thus fail to further improve many real-world applications by exploring the geometric structures of multi-view data. To address the above-mentioned issue, we introduce hierarchical graph augmented stacked autoencoders (HGSAE) for unsupervised multi-view representation learning. Specifically, a hierarchical graph structure is first adapted to stacked autoencoders to learn view-specific representations, aiming to preserve the geometric information of multi-view data through local and non-local graph regularizations. A general or common representation can then be learned by reconstructing each single view using fully connected neural networks. By doing this, the proposed method not only preserves the geometric information in multi-view data but also automatically balances the complementarity/consistency among different views. Extensive experiments on six popular unsupervised representation learning datasets demonstrate the effectiveness of our method when compared with recent state-of-the-art autoencoder methods. • Design local and non-local graph constructions for view-specific representations. • Adapt hierarchical graph structure to stacked autoencoders. • Propose hierarchical graph augmented stacked autoencoders. [ABSTRACT FROM AUTHOR]

In recent years, deep neural networks have been successful in both industry and academia, especially for computer vision tasks. The great success of deep learning is mainly due to its scalability to encode large-scale data and to maneuver billions of model parameters. However, it is a challenge to deploy these cumbersome deep models on devices with limited resources, e.g., mobile phones and embedded devices, not only because of the high computational complexity but also the large storage requirements. To this end, a variety of model compression and acceleration techniques have been developed. As a representative type of model compression and acceleration, knowledge distillation effectively learns a small student model from a large teacher model. It has received rapid increasing attention from the community. This paper provides a comprehensive survey of knowledge distillation from the perspectives of knowledge categories, training schemes, teacher–student architecture, distillation algorithms, performance comparison and applications. Furthermore, challenges in knowledge distillation are briefly reviewed and comments on future research are discussed and forwarded. [ABSTRACT FROM AUTHOR]

Graph embedding plays an important role in dimensionality reduction for processing the high-dimensional data. In graph embedding, its keys are the different kinds of graph constructions that determine the performance of dimensionality reduction. Inspired by this fact, in this article we propose a novel graph embedding method named the double graphs-based discriminant projections (DGDP) by integrating two designed discriminative global graph constructions. The proposed DGDP can well discover the discriminant and geometrical structures of the high-dimensional data through the informative graph constructions. In two global graph constructions, we consider the geometrical distribution of each point on each edge of the graphs to define the adjacent weights with class information. Moreover, in the weight definition of one graph construction, we further strengthen pattern discrimination among all the classes to design the weights of the corresponding adjacent graph. To demonstrate the effectiveness of the proposed DGDP, we experimentally compare it with the state-of-the-art graph embedding methods on several data sets. The experimental results show that the proposed graph embedding method outperforms the competing methods with more power of data representation and pattern discrimination in the embedded subspace. [ABSTRACT FROM AUTHOR]

Collaborative representation-based classification (CRC) is one of the famous representation-based classification methods in pattern recognition. However, a testing sample in most of the CRC variants is collaboratively reconstructed by a linear combination of all the training samples from all the classes, the training samples from the class that the testing sample belongs to have no advantage in discriminatively and competitively representing and classifying the testing sample. Moreover, the incorrect classification can easily come into being when the training samples from the different classes are very similar. To address the issues, we propose a novel discriminative collaborative representation-based classification (DCRC) method via l 2 regularizations to enhance the power of pattern discrimination. In the proposed model, we consider not only the discriminative decorrelations among all the classes, but also the similarities between the reconstructed representation of all the classes and the class-specific reconstructed representations in the l 2 regularizations. The experiments on several public face databases have demonstrated that the proposed DCRC effectively and robustly outperforms the state-of-the-art representation-based classification methods. [ABSTRACT FROM AUTHOR]

Collaborative representation-based classification (CRC) is a famous representation-based classification method in pattern recognition. Recently, many variants of CRC have been designed for many classification tasks with the good classification performance. However, most of them ignore the inter-class pattern discrimination among the class-specific representations, which is very critical for strengthening the pattern discrimination of collaborative representation (CR). In this article, we propose a novel CR approach for image classification, called weighted discriminative collaborative competitive representation (WDCCR). The proposed WDCCR designs the discriminative and competitive collaborative representation among all the classes by fully considering the class information. On the one hand, we incorporate two discriminative constraints into the unified WDCCR model. Both constraints are the competitive class-specific representation residuals and the pairs of class-specific representations for each query sample. On the other hand, the constraint of the weighted categorical representation coefficients is introduced into the proposed model for further enhancing the power of discriminative and competitive representation. In the weighted constraint, we assume that the different classes of each query sample should have less contribution to the representation with the small representation coefficients, and then two types of weight factors are designed to constrain the representation coefficients. Furthermore, the robust WDCCR (R-WDCCR) is proposed with l 1 -norm representation fidelity for recognizing noisy images. Extensive experiments on six image data sets demonstrate the effective and robust superiorities of the proposed WDCCR and R-WDCCR over the related state-of-the-art representation-based classification methods. • Propose two WDCCR methods with l 2 -norm fidelity. • Propose two robust WDCCR methods with l 1 -norm fidelity. • Design two weight factors in the proposed WDCCR methods. [ABSTRACT FROM AUTHOR]

• Introduce the coarse to fine probabilistic collaborative representation. • Propose two-phase PCRC method with l1-norm and l2-norm fidelities. • Propose two-phase weighted PCRC method with l1-norm and l2-norm fidelities. The probabilistic collaborative representation-based classification (PCRC), as a novel extension of collaborative representation-based classification (CRC), is a promising method in pattern recognition. In this article, we adopt the coarse to fine representation to propose two-phase probabilistic collaborative representation based-classification (TPCRC) to enhance the power of pattern discrimination in PCRC. In TPCRC, the first phase is to utilize probabilistic collaborative representation to coarsely choose the nearest representative samples, and the second phase is to use the chosen nearest samples to finely represent and classify each testing sample. In order to employ the locality of data to further improve classification performance of PCRC, we also propose two-phase weighted probabilistic collaborative representation based-classification (TWPCRC). In the fine representation of TWPCRC, the probabilistic collaborative coefficients are weighted by the local distance similarities between each testing sample and all the training samples. The proposed methods are verified by the comparative experiments on three public image data sets. Experimental results show that the proposed methods outperform the state-of-the-art collaborative representation-based classification methods. [ABSTRACT FROM AUTHOR]

Collaborative representation (CR) is one of the well-known representation methods and has been widely used in computer vision and pattern recognition. The collaborative representation-based classification (CRC) and its extension called the probabilistic collaborative representation-based classification (PCRC) have obtained promising performance in image classification. However, the representation fidelity is usually measured by the ℓ 2 -norm , which is not robust to outliers. Moreover, CRC and PCRC only consider the global distribution of data and ignore the locality of data. To overcome those problems, in this paper, we propose the residual-based extensions and the weighted version of CRC and PCRC. Specifically, for the residual-based extensions of CRC and PCRC, we measure the representation fidelity by jointing ℓ 1 -norm and the ℓ 2 -norm of coding residuals. For the weighted extensions of CRC and PCRC with different coding residuals, we constrained the representation coefficients with locality of data. To verify the effectiveness of the proposed methods, we conduct extensive experiments on six popular face databases and three image databases. Experimental results have demonstrated that the ℓ 1 -norm of coding residual, jointing both the ℓ 1 -norm and the ℓ 2 -norm of coding residuals and the locality constraint of representation coefficients can enhance pattern discrimination effectively. [ABSTRACT FROM AUTHOR]

Knowledge distillation (KD) is a widely adopted model compression technique for improving the performance of compact student models, by utilizing the "dark knowledge" of a large teacher model. However, previous studies have not adequately investigated the effectiveness of supervision from the teacher model, and overconfident predictions in the student model may degrade its performance. In this work, we propose a novel framework, Teacher–Student Complementary Sample Contrastive Distillation (TSCSCD), that alleviate these challenges. TSCSCD consists of three key components: Contrastive Sample Hardness (CSH), Supervision Signal Correction (SSC), and Student Self-Learning (SSL). Specifically, CSH evaluates the teacher's supervision for each sample by comparing the predictions of two compact models, one distilled from the teacher and the other trained from scratch. SSC corrects weak supervision according to CSH, while SSL employs integrated learning among multi-classifiers to regularize overconfident predictions. Extensive experiments on four real-world datasets demonstrate that TSCSCD outperforms recent state-of-the-art knowledge distillation techniques. • We present a supervision correction component for knowledge distillation (KD). • We propose a self-learning component that constrains the overconfident prediction. • The effectiveness of framework is evaluated on four real-world datasets. [ABSTRACT FROM AUTHOR]

K-nearest neighbor classification method (KNN), as one of the top 10 algorithms in data mining, is a very simple and yet effective nonparametric technique for pattern recognition. However, due to the selective sensitiveness of the neighborhood size k, the simple majority vote, and the conventional metric measure, the KNN-based classification performance can be easily degraded, especially in the small training sample size cases. In this article, to further improve the classification performance and overcome the main issues in the KNN-based classification, we propose a local mean representation-based k-nearest neighbor classifier (LMRKNN). In the LMRKNN, the categorical k-nearest neighbors of a query sample are first chosen to calculate the corresponding categorical k-local mean vectors, and then the query sample is represented by the linear combination of the categorical k-local mean vectors; finally, the class-specific representation-based distances between the query sample and the categorical k-local mean vectors are adopted to determine the class of the query sample. Extensive experiments on many UCI and KEEL datasets and three popular face databases are carried out by comparing LMRKNN to the state-of-art KNN-based methods. The experimental results demonstrate that the proposed LMRKNN outperforms the related competitive KNN-based methods with more robustness and effectiveness. [ABSTRACT FROM AUTHOR]

Abstract K-nearest neighbor rule (KNN) is one of the most widely used methods in pattern recognition. However, the KNN-based classification performance is severely affected by the sensitivity of the neighborhood size k and the simple majority voting in the regions of k -neighborhoods, especially in the case of the small sample size with the existing outliers. To overcome the issues, we propose two locality constrained representation-based k -nearest neighbor rules with the purpose of further improving the KNN-based classification performance. The one is the weighted representation-based k -nearest neighbor rule (WRKNN). In the WRKNN, the test sample is represented as the linear combination of its k -nearest neighbors from each class, and the localities of k -nearest neighbors per class as the weights constrain their corresponding representation coefficients. Using the representation coefficients of k -nearest neighbors per class, the representation-based distance between the test sample and the class-specific k -nearest neighbors is calculated as the classification decision rule. The other one is the weighted local mean representation-based k -nearest neighbor rule (WLMRKNN). In the WLMRKNN, k -local mean vectors of k -nearest neighbors per class are first calculated and then used for representing the test sample. In the linear combination of the class-specific k -local mean vectors to represent the test sample, the localities of k -local mean vectors per class are considered as the weights to constrain the representation coefficients of k -local mean vectors. The representation coefficients are employed to design the classification decision rule which is the class-specific representation-based distance between the test sample and k -local mean vectors per class. To demonstrate the effectiveness of the proposed methods, we conduct extensive experiments on the UCI and UCR data sets and face databases, in comparisons with seven related competitive KNN-based methods. The experimental results show that the proposed methods perform better with less sensitiveness to k , especially in the small sample size cases. [ABSTRACT FROM AUTHOR]

Highlights • Propose GMDKNN method. • Design categorical multi-generalized mean distances. • Design categorical nested generalized mean distance for classification. Abstract K -nearest neighbor (KNN) rule is a well-known non-parametric classifier that is widely used in pattern recognition. However, the sensitivity of the neighborhood size k always seriously degrades the KNN-based classification performance, especially in the case of the small sample size with the existing outliers. To overcome this issue, in this article we propose a generalized mean distance-based k -nearest neighbor classifier (GMDKNN) by introducing multi-generalized mean distances and the nested generalized mean distance that are based on the characteristic of the generalized mean. In the proposed method, multi-local mean vectors of the given query sample in each class are calculated by adopting its class-specific k nearest neighbors. Using the achieved k local mean vectors per class, the corresponding k generalized mean distances are calculated and then used to design the categorical nested generalized mean distance. In the classification phase, the categorical nested generalized mean distance is used as the classification decision rule and the query sample is classified into the class with the minimum nested generalized mean distance among all the classes. Extensive experiments on the UCI and KEEL data sets, synthetic data sets, the KEEL noise data sets and the UCR time series data sets are conducted by comparing the proposed method to the state-of-art KNN-based methods. The experimental results demonstrate that the proposed GMDKNN performs better and has the less sensitiveness to k. Thus, our proposed GMDKNN with the robust and effective classification performance could be a promising method for pattern recognition in some expert and intelligence systems. [ABSTRACT FROM AUTHOR]

Sparse representation-based classification (SRC) and collaborative representation-based classification (CRC) have shown promising classification results. Both methods are distance-based classifiers, and they represent a test sample with coefficients solved by different sparsity regularizations. The reason why the representation coefficient vector can be sparse is that the test sample can be represented by only a small subset of the whole dictionary, which means that only a few entries of the coefficient vector have nonzero values. Previous studies show that sparsity of representation coefficients of SRC and CRC is significant for achieving good classification performance. However, the parameter of the algorithm is closely associated with the sparsity and it is very hard to obtain the optimal parameter value. In this paper, we propose two novel versions of SRC and CRC and implement four algorithms. The first version has two implementation named SQ_SRC and SQ_CRC, which use squared value of each entry of the representation coefficient vector to enhance sparsity of representation coefficients. SQ_SRC and SQ_CRC allow us to obtain more robust representation and classification performance of the test sample. The second version has another two implementation named SQF_SRC and SQF_CRC, which integrate original SRC and CRC with the SQ_SRC and SQ_CRC to perform classification. We conduct extensive experiments on a number of facial datasets, as well as a couple of non-facial datasets. The experimental results demonstrated that our methods output much more promising classification results than naive SRC and CRC in most scenarios. [ABSTRACT FROM AUTHOR]

Sparse representation methods based on l 1 and/or l 2 regularization have shown promising performance in different applications. Previous studies show that the l 1 regularization based representation has more sparse property, while the l 2 regularization based representation is much simpler and faster. However, when dealing with noisy data, both naive l 1 and l 2 regularization suffer from the issue of unsatisfactory robustness. In this paper, we explore the method to implement an antinoise sparse representation method for robust face recognition based on a joint version of l 1 and l 2 regularization. The contributions of this paper are mainly shown in the following aspects. First, a novel objective function combining both l 1 and l 2 regularization is proposed to implement an antinoise sparse representation. An iterative fitting operation via l 1 regularization is integrated with l 2 norm minimization, to obtain an antinoise classification. Second, the rationale how the proposed method produces promising discriminative and antinoise performance for face recognition is analyzed. The l 2 regularization enhances robustness and runs fast, and l 1 regularization helps cope with the noisy data. Third, the classification robustness of the proposed method is demonstrated by extensive experiments on several benchmark facial datasets. The method can be considered as an option for the expert systems for biometrics and other recognition problems facing unstable and noisy data. [ABSTRACT FROM AUTHOR]

K -nearest neighbor rule (KNN) has been regarded as one of the top 10 methods in the field of data mining. Due to its simplicity and effectiveness, it has been widely studied and applied to various classification tasks. In this article, we develop a novel representation coefficient-based k -nearest centroid neighbor method (RCKNCN), which aims to further improve the classification performance and reduce the method's sensitivity to the neighborhood size k , especially in the cases of small sample size. Different from existing KNN-based methods, RCKNCN is able to capture both the proximity and the geometry of k -nearest neighbors, and learn to differentiate the contribution of each neighbor to the classification of a testing sample through a linear representation method. Moreover, under the RCKNCN framework, we also propose a novel weighted majority voting algorithm using the representation coefficients associated with individual nearest centroid neighbors, which are deemed to hold more discriminative information of the neighbors. To fully study the classification performance of RCKNCN, we compare it with the state-of-the-art KNN-based methods on many data sets that are widely used in the literature. The extensive experiments demonstrate the effectiveness and robustness of our method in various classification tasks. • Propose RCKNCN based on both NCN and representation of neighbors. • Differentiate the contribution of each centroid neighbor via representation. • Design a new representation coefficient-based majority voting decision. [ABSTRACT FROM AUTHOR]

k -Nearest neighbor (KNN) rule is a very simple and powerful classification algorithm. In this article, we propose a new KNN-based classifier, called the local mean-based pseudo nearest neighbor (LMPNN) rule. It is motivated by the local mean-based k -nearest neighbor (LMKNN) rule and the pseudo nearest neighbor (PNN) rule, with the aim of improving the classification performance. In the proposed LMPNN, the k nearest neighbors from each class are searched as the class prototypes, and then the local mean vectors of the neighbors are yielded. Subsequently, we attempt to find the local mean-based pseudo nearest neighbor per class by employing the categorical k local mean vectors, and classify the unknown query patten according to the distances between the query and the pseudo nearest neighbors. To assess the classification performance of the proposed LMPNN, it is compared with the competing classifiers, such as LMKNN and PNN, in terms of the classification error on thirty-two real UCI data sets, four artificial data sets and three image data sets. The comprehensively experimental results suggest that the proposed LMPNN classifier is a promising algorithm in pattern recognition. [ABSTRACT FROM AUTHOR]

In this article, we develop a linear supervised subspace learning method called locality-based discriminant neighborhood embedding (LDNE), which can take advantage of the underlying submanifold-based structures of the data for classification. Our LDNE method can simultaneously consider both ‘locality’ of locality preserving projection (LPP) and ‘discrimination’ of discriminant neighborhood embedding (DNE) in manifold learning. It can find an embedding that not only preserves local information to explore the intrinsic submanifold structure of data from the same class, but also enhances the discrimination among submanifolds from different classes. To investigate the performance of LDNE, we compare it with the state-of-the-art dimensionality reduction techniques such as LPP and DNE on publicly available datasets. Experimental results show that our LDNE can be an effective and robust method for classification. [ABSTRACT FROM PUBLISHER]

K-nearest neighbor (KNN) rule is a simple and effective algorithm in pattern classification. In this article, we propose a local mean-based k-nearest centroid neighbor classifier that assigns to each query pattern a class label with nearest local centroid mean vector so as to improve the classification performance. The proposed scheme not only takes into account the proximity and spatial distribution of k neighbors, but also utilizes the local mean vector of k neighbors from each class in making classification decision. In the proposed classifier, a local mean vector of k nearest centroid neighbors from each class for a query pattern is well positioned to sufficiently capture the class distribution information. In order to investigate the classification behavior of the proposed classifier, we conduct extensive experiments on the real and synthetic data sets in terms of the classification error. Experimental results demonstrate that our proposed method performs significantly well, particularly in the small sample size cases, compared with the state-of-the-art KNN-based algorithms. [ABSTRACT FROM PUBLISHER]

Representation-based classification (RBC) has attracted much attention in pattern recognition. As a linear representative RBC method, collaborative representation-based classification (CRC) is very promising for classification. Although many extensions of CRC have been developed recently, the discriminative and competitive representations of different classes for favorable classification has not yet been fully explored. To design the discriminative and competitive collaborative representations for enhancing the power of pattern discrimination, we propose a novel double competitive constraints-based collaborative representation for classification (DCCRC). In the proposed DCCRC, one competitive constraint is the l 2 -norm regularization of residuals between each query sample and the class-specific representations, the other one is the l 2 -norm regularization of the representations of all the classes excluding any one class. In two competitive constraints, the class discrimination information is employed to generate competitive representations. Moreover, the proposed method integrates both the representation learning and classification into the unified model. We study the effectiveness and robustness of the proposed method by comparing it with the state-of-the-art CRC methods on six face databases and twelve UCI data sets. The experimental results demonstrate the promising classification performance of the proposed method. [ABSTRACT FROM AUTHOR]

• Propose the DGLPGE method. • Design new local and global graph constructions. • Characterize locality and globality preserving scatters. • It can capture the geometry of data and enhance pattern discrimination. Graph embedding in dimensionality reduction has attracted much attention in the high-dimensional data analysis. Graph construction in graph embedding plays an important role in the quality of dimensionality reduction. However, the discrimination information and the geometrical distributions of data samples are not fully exploited for discovering the essential geometrical and discriminant structures of data and strengthening the pattern discrimination in graph constructions of graph embedding. To overcome the limitations of graph constructions, in this article we propose a novel graph-based dimensionality reduction method entitled discriminative globality and locality preserving graph embedding (DGLPGE) by designing the informative globality and locality preserving graph constructions. In the constructed graphs, bidirectional weights of edges are newly defined by considering both the geometrical distributions of each point of edges and the class discrimination. Using the adjacent weights of graphs, we characterize the intra-class globality preserving scatter, the inter-class globality preserving scatter and the locality preserving scatter to formulate the objective function of DGLPGE in order to optimize the projection of dimensionality reduction. Extensive experiments demonstrate that the proposed DGLPGE often outperforms the state-of-the-art dimensionality reduction methods. [ABSTRACT FROM AUTHOR]

• Propose the WELRMC method. • Propose the TPWELRMC method. • Design weighted sparsity augmented collaborative representation. • Design coarse to fine augmented representation. Representation-based classification (RBC) methods have recently been the promising pattern recognition technologies for object recognition. The representation coefficients of RBC as the linear reconstruction measure (LRM) can be well used for classifying objects. In this article, we propose two enhanced linear reconstruction measure-based classification methods based on the sparsity-augmented collaborative representation-based classification method (SA-CRC). One is the weighted enhancement linear reconstruction measure-based classification method (WELRMC) that introduces data localities into SA-CRC. Another is the two-phase weighted enhancement linear reconstruction measure-based classification method (TPWELRMC) that integrates both the coarse and fine representations into SA-CRC. To demonstrate the effectiveness of the proposed methods, experiments are conducted on several public face databases in comparison with the state-of-the-art representation-based classification methods. The experimental results show that the proposed methods significantly outperform the competing RBC methods. [ABSTRACT FROM AUTHOR]

Deep neural networks have achieved a great success in a variety of applications, such as self-driving cars and intelligent robotics. Meanwhile, knowledge distillation has received increasing attention as an effective model compression technique for training very efficient deep models. The performance of the student network obtained through knowledge distillation heavily depends on whether the transfer of the teacher's knowledge can effectively guide the student training. However, most existing knowledge distillation schemes require a large teacher network pre-trained on large-scale data sets, which can increase the difficulty of knowledge distillation in different applications. In this article, we propose a feature fusion-based collaborative learning for knowledge distillation. Specifically, during knowledge distillation, it enables networks to learn from each other using the feature/response-based knowledge in different network layers. We concatenate the features learned by the teacher and the student networks to obtain a more representative feature map for knowledge transfer. In addition, we also introduce a network regularization method to further improve the model performance by providing a positive knowledge during training. Experiments and ablation studies on two widely used data sets demonstrate that the proposed method, feature fusion-based collaborative learning, significantly outperforms recent state-of-the-art knowledge distillation methods. [ABSTRACT FROM AUTHOR]

Point cloud completion is prevalent due to the insufficient results from current point cloud acquisition equipments, where a large number of point data failed to represent a relatively complete shape. Existing point cloud completion algorithms, mostly encoder‐decoder structures with grids transform (also presented as folding operation), can hardly obtain a persuasive representation of input clouds due to the issue that their bottleneck‐shape result cannot tell a precise relationship between the global and local structures. For this reason, this article proposes a novel point cloud completion model based on a Stack‐Style Folding Network (SSFN). Firstly, to enhance the deep latent feature extraction, SSFN enhances the exploitation of shape feature extractor by integrating both low‐level point feature and high‐level graphical feature. Next, a precise presentation is obtained from a high dimensional semantic space to improve the reconstruction ability. Finally, a refining module is designed to make a more evenly distributed result. Experimental results shows that our SSFN produces the most promising results of multiple representative metrics with a smaller scale parameters than current models. [ABSTRACT FROM AUTHOR]

Face recognition in real-world video surveillance needs to deal with a lot of challenges including low resolution, illumination variations, pose changes, occlusions and so on. Among them, occlusions are difficult and have not attracted enough attentions. To address this problem, in this paper, we propose a robust discriminative nonnegative dictionary learning method for occluded face recognition, which estimates the occlusions adaptively and selects the features robustly. Instead of modeling the reconstruction errors using a specific distribution, we estimate occlusions adaptively according to the reconstruction errors and learn different weights for different pixels during the iterative processing. To enhance discriminant ability of the dictionary, we constrain the low-dimensional representations of samples from the same class to be as close as possible and select the discriminative features robustly via ℓ 2, 1 -norm. For the induced non-convex problem, we reformulate it into local convex optimization subproblem via utilizing the half-quadratic technique and propose new update rules. Extensive experiments are implemented on four benchmark datasets, and the experimental results demonstrate the effectiveness and robustness of the proposed method. [ABSTRACT FROM AUTHOR]

Feature space-based face recognition method performs representation and classification for face images in the feature space instead of the original space. This representation-based method uses kernel function to solve the feature space and therefore outperforms the method working in original space. A contemporary coarse-to-fine face recognition method also uses the idea of sparse representation but separates the classification into two stages from coarse candidate to fine classification. They both apply the sparse representation technique for image classification, but we believe that there is still space to improve the robustness of sparse representation in these methods. We proposed a novel method integrating both methods together to perform robust face image classification. This method also consists of two stages as sparse representation-based coarse-to-fine classification. In the first stage, however, the coarse candidate classes are determined in the feature space selected by the kernel function instead, which is the key technique used in the feature space-based method. And then in the next stage, fine classification is performed to assign the test sample into a class from candidate classes. The experimental results demonstrated that this novel integrated method outperforms both methods on prevailing benchmark facial datasets. [ABSTRACT FROM AUTHOR]

Graph neural networks (GNNs) have garnered significant attention for their ability to effectively process graph-related data. Most existing methods assume that the input graph is noise-free; however, this assumption is frequently violated in real-world scenarios, resulting in impaired graph representations. To address this issue, we start from the essence of graph structure learning, considering edge discovery and removal, reweighting of existing edges, and differentiability of the graph structure. We introduce virtual nodes and consider connections with virtual nodes to generate optimized graph structures, and subsequently utilize Gumbel-Softmax to reweight edges and achieve differentiability of the Graph Structure Learning (VN-GSL for abbreviation). We conducted a thorough evaluation of our method on a range of benchmark datasets under both clean and adversarial circumstances. The results of our experiments demonstrate that our approach exhibits superiority in terms of both performance and efficiency. Our implementation will be made available to the public. [ABSTRACT FROM AUTHOR]

Video semantic detection has been one research hotspot in the field of human-computer interaction. In video features-oriented sparse representation, the features from the same category video could not achieve similar coding results. To address this, the Locality-Sensitive Discriminant Sparse Representation (LSDSR) is developed, in order that the video samples belonging to the same video category are encoded as similar sparse codes which make them have better category discrimination. In the LSDSR, a discriminative loss function based on sparse coefficients is imposed on the locality-sensitive sparse representation, which makes the optimized dictionary for sparse representation be discriminative. The LSDSR for video features enhances the power of semantic discrimination to optimize the dictionary and build the better discriminant sparse model. More so, to further improve the accuracy of video semantic detection after sparse representation, a weighted K-Nearest Neighbor (KNN) classification method with the loss function that integrates reconstruction error and discrimination for the sparse representation is adopted to detect video semantic concepts. The proposed methods are evaluated on the related video databases in comparison with existing sparse representation methods. The experimental results show that the proposed methods significantly enhance the power of discrimination of video features, and consequently improve the accuracy of video semantic concept detection. [ABSTRACT FROM AUTHOR]

Because of the Student- t distribution owning heavier tailed than the Gaussian distribution, under a Bayesian framework, a spatially variant finite mixture model with Student’s t -distribution component function is proposed for grayscale image segmentation. To avoid additional computational step and improve the efficiency of the proposed model, a representation of contextual mixing proportion is adopted. Secondly, the spatial information of the pixels is closely related to the Gaussian distribution of their neighborhood system. Thirdly, the inherent relationship between the Gaussian distribution and the Student’s t -distribution is adopted to optimize the unknown parameters of the proposed model, which simplifies the inference process and makes the proposed model to be easily implemented. Comprehensive experiments on synthetic noise images, simulated medical images and real-world grayscale images are presented to illustrate the superior performance of the proposed model in terms of the visual and quantitative comparison. [ABSTRACT FROM AUTHOR]

Most of the previous regression models for predicting the remaining useful life (RUL) of hard disks primarily focused on failure data obtained through SMART monitoring. However, models that only considered failure data were not accurate enough. In order to address the issue of imbalanced sample quantities in different categories within the dataset, this paper proposes a method for predicting the RUL of hard disks under imbalanced conditions. It takes into account the imbalance issues in both SMART failure data and healthy data, as well as the imbalance in monitoring data attributes between hard disk drives and solid-state drives. The paper improves upon the traditional Transformer time series model and constructs a comprehensive and unified model for the RUL of hard disks across multiple categories. The proposed model achieves an RMSE score of 3.325, and has 66 % in MAE improved to the conventional models. The proposed model has promising prospects for future applications in predicting the remaining life of hard disks. [ABSTRACT FROM AUTHOR]

Platform merchants mine user clusters and their characteristics to assist in precision marketing. In view of the information overload in e-commerce reviews, machine methods are needed to efficiently obtain clustering information from text. This study innovatively integrated the emotional correlation analysis model and Self-organizing Map (SOM) in application, to construct fine-grained user emotion vector based on review text and perform visual cluster analysis, which helped quickly mine user clustering and characteristics from review text. The result of empirical analysis based on real reviews of Amazon books showed that the proposed method had the average precision as 0.71, confirming that the clustering method integrating the emotional correlation analysis model and SOM could efficiently mine user groups and match appropriate marketing strategies, which will help platform merchants carry out precision marketing. The study makes contributions to the application and innovation of researches in the field of user clustering and e-commerce precision marketing. [ABSTRACT FROM AUTHOR]

In multi-label classification, challenges arise from missing labels due to subjective analysis or label ambiguity. This makes it difficult to accurately capture label correlations and enhance classifier performance. Previous research has primarily focused on obtaining label correlations with sparsity assumptions from incomplete label matrices to fill in these missing labels. However, this approach leads to inaccurate label correlations since it relies on incomplete label matrix information. To overcome this issue and improve accuracy, this paper proposes a new method called LCFM, which effectively learns label correlations using a combination of global and local label-specific features to recover missing labels. Firstly, we rely on the sparsity assumption of Lasso regression coefficients to select label-specific features from the training samples. Secondly, we introduce l 2 -norm regularization to learn global label correlations through global label-specific features. simultaneously, we utilize similar label-specific features to describe local label correlations. Finally, we minimize the difference between predicted results and true labels using the binary cross-entropy loss function and employ the Accelerated Proximal Gradient (APG) algorithm for model optimization. Experimental results comparing LCFM with five benchmark indicators on nine datasets demonstrate that our proposed method outperforms other advanced methods, showing strong competitiveness. [ABSTRACT FROM AUTHOR]

Non-contact facial video-based physiological signal measurement using remote photoplethysmography (rPPG) has shown great potential in many applications (e.g. telemedicine, driver fatigue detection, sleep detection) and has achieved reliable results in the controlled environment. With the development of deep learning, more and more methods have been proposed. However, these methods were usually implemented based on model variations that have achieved better results in computer vision tasks, and finally chose average pooling layer or one-dimensional convolution layer directly reduce dimension. Such these operations ignored the fact that the physiological signals of one individual are consistent in different facial regions. In this paper, we propose a novel remote photoplethysmography framework that utilizes graph neural network to extract physiological signal, named GraphPhys. And we design the Average Relative GraphConv based on the fact that the physiological signals of one individual are always consistent in different facial regions for the remote physiological signal measurement task. We apply GraphPhys to two different architectures for rPPG methods, and the methods based on GraphPhys significantly outperform the original methods on the VIPL-HR (Niu et al., 2019) dataset. To the best of our knowledge, this is the first work to utilize graph neural network to aggregate physiological signals. The code is available at https://github.com/Xiong-JiaHao/GraphPhys. [ABSTRACT FROM AUTHOR]

Cross-modal retrieval aims to search the semantically similar instances from the other modalities by giving a query from one modality. Recently, generative adversarial networks (GANs) has been proposed to model the joint distribution over the data from different modalities and to learn the common representations for cross-modal retrieval. However, most of existing GANs-based methods simply project original representations of different modalities into a common representation space, and ignore the fact that different modalities share the common characteristics and on the other side each modality has the individual characteristics. To address this problem, in this paper, we propose a novel cross-modal retrieval method, called representation separation adversarial networks, which explicitly separates the original representations into common latent representations and private representations. Specifically, we minimize the correlation between the common representations and private representations to ensure independence of them. Then, we reconstruct the original representations via exchanging the common representations of different modalities to encourage the information swap. Finally, the labels are utilized to increase the discriminant of common representations. Comprehensive experimental results on two widely used datasets show that the proposed method achieved better performance than many existing GANs-based methods, and demonstrate that explicitly modeling the private representation for each modality can improve the model to extract common latent representations. [ABSTRACT FROM AUTHOR]

Visual Question Answering (VQA) is an increasingly popular research area in machine learning. Most of the existing VQA tasks only focus on static images, and only a few models are based on videos. The primary purpose of this project is to develop an innovative model that performs Affective Question Answering on Video (AQAV), a multi-tasking architecture that implements a Video QA route and an Affective route. A pre-trained CNN emotion detector recognizes emotions on the frames of a video, and a string of the emotion labels is relayed to the Token-based, Frame-based and Integrated attention mechanisms. The attention model uses the visual features, the question and the emotion labels to focus on relevant frames of the video and relevant regions of the frames. The string of emotion labels is used to generate an emotion caption that will be used by the Text QA module to prepare an affective answer. A conventional answer is generated from processes that take place along the Video QA route, while the affective answer is a product of both the Video QA and the Affective routes. Our model does not only make VQA more analytic by generating an explanatory answer, but also registers quantitative improvement in performance, when compared with previous baselines. We managed to prove that the injection of emotions in the attention mechanism boosts VQA performance. The AQAV model contributes towards efforts in making machines understand sequential and dynamic visual scenes in the real world. [ABSTRACT FROM AUTHOR]

Collaborative representation-based classification has shown promising results on cognitive vision tasks like face recognition. It solves a linear problem with l 1 or l 2 norm regularization to obtain a stable sparse representation. Previous studies showed that the collaboration representation assisted the output of optimum sparsity constraint, but the choice of regularization also played a crucial role in stable representation. In this paper, we proposed a novel discriminative collaborative representation-based classification method via regularization implemented by truncated total least squares algorithm. The key idea of the proposed method is combining two coefficients obtained by l 2 regularization and truncated TLS-based regularization. After evaluated by extensive experiments conducted on several benchmark facial databases, the proposed method is demonstrated to outperform the naive collaborative representation-based method, as well as some other state-of-the-art methods for face recognition. The regularization by truncation effectively and dramatically enhances sparsity constraint on coding coefficients in collaborative representation and increases robustness for face recognition. [ABSTRACT FROM AUTHOR]

Subspace clustering has attracted much attention on account of its excellent performance in various tasks, such as computer vision and pattern recognition. Since the high-dimensional data may comprise redundant information and noises, low-dimensional subspace structure is difficult to be recovered exactly. Existing methods adopt the non-convex Schatten- p norm to approximate the rank function closer. However, these methods are still unable to handle each singular value flexibly, which may yield unsatisfactory solutions. To address this deficiency, in this paper, we propose a iterative reweighted Frobenius norm regularized latent low rank representation (IRFLLRR) model, which can restrain the small and large rank components simultaneously with rational weights. Specifically, the reweighting strategy can preserve the desirable structure information, while removing the sparse noise and redundant information. Hence the constructed coefficient matrix can capture the global structure of data more adequately. To solve the proposed model, an efficient algorithm is developed via the alternating direction method of multipliers (ADMM), which ensures each subproblem can be optimized in closed-form. In addition, we provide detailed mathematical proofs to guarantee that the sequence generated by our algorithm convergences to a Karush–Kuhn–Tucker (KKT) point. Extensive experimental results reveal the superiority of the proposed method over several state-of-the-art methods. MATLAB code is available at https://github.com/wangzhi-swu/IRFLLRR. • Propose a new subspace clustering model. • Develop an efficient algorithm to solve the proposed model. • Provide detailed mathematical proofs to analyze the proposed algorithm. [ABSTRACT FROM AUTHOR]

As an important task of artificial intelligence, natural language conversation has attracted wide attention of researchers in natural language processing. Existing works in this field mainly focus on consistency of neural response generation whilst ignoring the effect of emotion state on the response generation. In this paper, we propose an Emotion-based natural language Responding Model (ERM) to address the challenging issue in conversation. ERM encodes the emotion state of the conversation as distributed embedding into the process of response generation, redefines an objective function that jointly trains our model and introduces a novel re-rank function to select the appropriate response. Experimental results on Chinese conversation dataset show that our method yields qualitative performance improvements in the Perplexity (PPL), Word Error-rate (WER) and Bilingual Evaluation Understudy (BLEU) compared with the baseline sequence-to-sequence (Seq2Seq) model, and achieves better performance than the state-of-the-art in terms of emotion and content consistency of the response. [ABSTRACT FROM AUTHOR]

Abstract The concept of Visual Question Answering (VQA) has recently attracted the attention of many researchers in the field of machine learning. Different attention models have been proposed in VQA for the purpose of addressing the need to focus on local regions of an image. This paper proposes the concept of Mood-Aware Visual Question Answering (MAVQA) using novel long short term memory (LSTM) and convolutional neural network (CNN) attention models that combine the local image features, the question and the mood detected from the particular region of the image to produce a mood-based answer using a pre-processed image dataset. The attention mechanisms serve to enable the VQA model to only focus on parts of the image that are relevant to both the detected mood and the key words in the question. The irrelevant parts of the image are ignored, thus improving classification accuracy by reducing the chances of predicting wrong answers. Whereas previous efforts have utilized CNN mostly for the embedding of images and text, we formulate a CNN attention algorithm for the image, question and mood. The more direct convolutional attention operation is more efficient and effective, when the number of views and kernel length are optimized, than the winding recurrent LSTM attention operation. The experimental results prove that MAVQA is effectively mood-aware, and the accuracy levels of our LSTM attention model are well within the range of previous conventional VQA benchmarks, while our novel CNN attention model outperforms the previous baselines in several instances. The additional attention on the mood does not only improve classification accuracy but also substantially contributes towards the analysis and comprehension of image features, a key development in modern artificial intelligence. [ABSTRACT FROM AUTHOR]

With the development of medical informatization, a large number of patients' electronic medical records (EMRs) have been accumulated in the hospital information system, which is characterized by multi-structured data form, diversified professional vocabulary categories and fuzzy vocabulary demarcation. Natural language processing (NLP) provides a silver lining for parsing electronic medical records, and the mainstream methods include dictionary-based methods, rule-based and statistical methods, and machine learning methods. Due to the semantic richness and structural diversity of Chinese text, Chinese EMRs analysis methods are still scarce compared with English EMRs. In order to overcome the problems of unstructured, multiple meanings of words and unclear word boundaries in text of Chinese electronic medical records, this paper proposes a medical entity recognition method based on RoFormerV 2 -BiLSTM-CRF fusion model, using BIO annotation method to annotate the recognized medical entities, and using knowledge graph to analyze the medical entity relationships identified in single patient medical record, multiple patient medical records respectively. The relationships between the medical entities identified in a single patient record and multiple patient records are analyzed using knowledge graphs. The experimental analysis was conducted on the expert-annotated dataset CCKS2019, and the results showed that the proposed method was effective for recognizing "Diseases and Diagnoses", "Laboratory Tests", "Imaging Examinations", "Anatomical Sites", "Drugs" and "Surgery" in the dataset. The average accuracy, recall and F 1 -Score of the proposed method for the identification of the six medical entities, are 84.8%, 83.5% and 83.9%, respectively, which were 5.3%, 8.6% and 6.8% higher than the traditional Word2Vec-BiLSTM-CRF model, and the existing Word2Vec-BiLSTM-CRF and BERT-BiLSTM-CRF models were iteratively trained with the same evaluation. The experimental results show that the proposed model performs better in medical entity recognition. • Connect an optimized Transformer with the BiLSTM-CRF to form a medical entity recognition model. • Make Medical knowledge graphs based on a single medical record and multiple medical records. [ABSTRACT FROM AUTHOR]

This paper presents an angle and density-based data preprocessing method. It can be used to simultaneously identify outliers and boundary points (called uniformly boundary points). Detecting boundary points is often more interesting than detecting normal points, since they represent valid, interesting, and potentially valuable patterns. An efficient local geometry-based method is proposed for detecting such points by both angle and density measures. The unified measure is adaptive and stable by combining multiple features (angles and density), which can be used to evaluate to what degree a given point is a boundary point. Compared with two related state-of-the-art approaches, our method better reflects the characteristics of the data and provides similar but accuracies for more data set. Experimental results obtained for a number of synthetic and real-world data sets demonstrate the effectiveness and efficiency of our method. [ABSTRACT FROM AUTHOR]

Hematological parameters are key to reflect the health status of animals and their physiological adaptation to the environment. However, few studies focused on the inter- and intra-specific variations of hematological parameters in hynobiid salamanders. Here, we examined the hematological parameters of the stream salamander, Batrachuperus pinchonii, originating from two different altitudinal populations to explore their intra-specific variation. Sexual dimorphism is only present in the erythrocyte count and males have higher mean values than females. The morphometric values of erythrocyte, hemoglobin concentration, and erythrocyte count of the high altitudinal (Jiajin) population were smaller than those of the lower altitudinal (Sandaoping) population; however, a significant difference between two populations was only revealed in the case of erythrocyte length via ANOVA. The results of linear regression showed that a significant relationship was present between body condition and erythrocyte length as well as the erythrocyte length to erythrocyte width ratio. Our findings suggest that the features of hematological parameters in B. pinchonii are reflected in the size of erythrocyte, and neither in erythrocyte count nor in hemoglobin concentration. These results provide a foundation for assessing and monitoring the health status of this salamander species, and furthermore, for understanding the physiological basis of altitudinal adaptation. [ABSTRACT FROM AUTHOR]

Conventional approaches for Speech Emotion Recognition (SER) usually assume that the feature distributions between training and test set are identical. However, this assumption does not hold in many real scenarios. Although many Domain Adaptation (DA) methods have been proposed to solve this problem, the conventional emotion discriminative information is ignored. In this paper, we propose a DA based method called Emotion-discriminative and Domain-invariant Feature Learning Method (EDFLM) for SER, in which both the domain divergence and emotion discrimination are considered to learn emotion-discriminative and domain-invariant features by using emotion label constraint and domain label constraint. Furthermore, to disentangle the emotion-related factors from the emotion-unrelated factors, we introduce an orthogonal term to encourage the input to be disentangled into two blocks: emotion-related and emotion-unrelated features. Our method can learn emotion-discriminative and domain-invariant features through a back propagation network which uses the acoustic features of INTERSPEECH 2009 Emotion Challenge as the input rather than raw speech signals. Experiments on the INTERSPEECH 2009 Emotion Challenge two-class task show that the performance of our method is superior to other state-of-the-arts methods. [ABSTRACT FROM AUTHOR]

SMO (Sequential Minimal Optimization) is an outstanding SVM algorithm in efficiency and memory requirements. But it need cross validation to optimize parameters in the mathematical model to avoid the overfitting, which produces too much median classifiers, resulting in the decrease of the stability of algorithm and the increase of training time considerably. In this paper, by introducing the concept of “classification noise”, CNSMO (Classification Noise Detection based SMO algorithm) is proposed. In the CNSMO, an inseparable problem can be converted into a separable problem so that overfitting is not required to be taken into consideration. This makes cross validation can be avoided in the whole algorithm and the stability of the algorithm can be improved. The training time can be saved considerably. The penalty parameter in the CNSMO algorithm can be eliminated so that the model of the CNSMO is much simpler than other SMO algorithms. This paper presents that the proposed algorithm can reduce training time considerably without decreasing prediction accuracy. The effectiveness and efficiency of the CNSMO are demonstrated through experiments on public data sets. [ABSTRACT FROM AUTHOR]