Search

Facing with rapidly increasing demands for analyzing high-order data or multiway data, feature-extracting methods become imperative for analysis and processing. The traditional feature-extracting methods, however, either need to overly vectorize the data and smash the original structure hidden in data, such as PCA and PCA-like methods, which is unfavorable to the data recovery, or cannot eliminate the redundant information very well, such as tucker decomposition (TD) and TD-like methods. To overcome these limitations, we propose a more flexible and more powerful tool, called the multiview principal components analysis (Multiview-PCA) in this article. By segmenting a random tensor into equal-sized subarrays called sections and maximizing variations caused by orthogonal projections of these sections, the Multiview-PCA finds principal components in a parsimonious and flexible way. In so doing, two new operations on tensors, the S -direction inner/outer product, are introduced to formulate tensor projection and recovery. With different segmentation ways characterized by section depth and direction, the Multiview-PCA can be implemented many times in different ways, which defines the sequential and global Multiview-PCA, respectively. These multiple Multiview-PCA take the PCA and PCA-like, and TD and TD-like as the special cases, which correspond to the deepest section depth and the shallowest section depth, respectively. We propose an adaptive depth and direction selection algorithm for the implementation of Multiview-PCA. The Multiview-PCA is then tested in terms of subspace recovery ability, compression ability, and feature extraction performance when applied to a set of artificial data, surveillance videos, and hyperspectral imaging data. All numerical results support the flexibility, effectiveness, and usefulness of Multiview-PCA.

In most of the existing representation learning frameworks, the noise contaminating the data points is often assumed to be independent and identically distributed (i.i.d.), where the Gaussian distribution is often imposed. This assumption, though greatly simplifies the resulting representation problems, may not hold in many practical scenarios. For example, the noise in face representation is usually attributable to local variation, random occlusion, and unconstrained illumination, which is essentially structural, and hence, does not satisfy the i.i.d. property or the Gaussianity. In this article, we devise a generic noise model, referred to as independent and piecewise identically distributed (i.p.i.d.) model for robust presentation learning, where the statistical behavior of the underlying noise is characterized using a union of distributions. We demonstrate that our proposed i.p.i.d. model can better describe the complex noise encountered in practical scenarios and accommodate the traditional i.i.d. one as a special case. Assisted by the proposed noise model, we then develop a new information-theoretic learning framework for robust subspace representation through a novel minimum weighted error entropy criterion. Thanks to the superior modeling capability of the i.p.i.d. model, our proposed learning method achieves superior robustness against various types of noise. When applying our scheme to the subspace clustering and image recognition problems, we observe significant performance gains over the existing approaches.

Lately, variational autoencoder (VAE) has arisen as one of the most prevalent methods for unsupervised learning of intricate distributions. Despite being successful in deep feature extraction and uncertain data modelling, it still suffers from the instability and reconstruction error due to random sampling in the latent subspace representation of original input space. In this paper, to deal with those limitations, constrained VAE (CVAE) is proposed by utilizing input sample information. Enthused by parallel interaction mechanism between the ventral and dorsal stream of human brain in object recognition, parallel interaction spatial-temporal CVAE (PIST-CVAE) is proposed to extract spatial and temporal features from input samples. Lower dimensional nonlinear features extracted from PIST-CVAE are used to build the soft sensor. The effectiveness of CVAE and PIST-CVAE is demonstrated in an industrial case study, a polyester polymerization process.

Linear discriminant analysis (LDA) is a well-known technique for supervised dimensionality reduction and has been extensively applied in many real-world applications. LDA assumes that the samples are Gaussian distributed, and the local data distribution is consistent with the global distribution. However, real-world data seldom satisfy this assumption. To handle the data with complex distributions, some methods emphasize the local geometrical structure and perform discriminant analysis between neighbors. But the neighboring relationship tends to be affected by the noise in the input space. In this research, we propose a new supervised dimensionality reduction method, namely, locality adaptive discriminant analysis (LADA). In order to directly process the data with matrix representation, such as images, the 2-D LADA (2DLADA) is also developed. The proposed methods have the following salient properties: 1) they find the principle projection directions without imposing any assumption on the data distribution; 2) they explore the data relationship in the desired subspace, which contains less noise; and 3) they find the local data relationship automatically without the efforts for tuning parameters. The performance of dimensionality reduction shows the superiorities of the proposed methods over the state of the art.

The searching ability of the population-based search algorithms strongly relies on the coordinate system on which they are implemented. However, the widely used coordinate systems in the existing multifactorial optimization (MFO) algorithms are still fixed and might not be suitable for various function landscapes with differential modalities, rotations, and dimensions; thus, the intertask knowledge transfer might not be efficient. Therefore, this article proposes a novel intertask knowledge transfer strategy for MFOs implemented upon an active coordinate system that is established on a common subspace of two search spaces. The proper coordinate system might identify some common modality in a proper subspace to some extent. In this article, to seek the intermediate subspace, we innovatively introduce the geodesic flow that starts from a subspace, reaching another subspace in unit time. A low-dimension intermediate subspace is drawn from a uniform distribution defined on the geodesic flow, and the corresponding coordinate system is given. The intertask trial generation method is applied to the individuals by first projecting them on the low-dimension subspace, which reveals the important invariant features of the multiple function landscapes. Since intermediate subspace is generated from the major eigenvectors of tasks' spaces, this model turns out to be intrinsically regularized by neglecting the minor and small eigenvalues. Therefore, the transfer strategy can alleviate the influence of noise led by redundant dimensions. The proposed method exhibits promising performance in the experiments.

In this article, we propose a novel feature selection approach via explicitly addressing the long-standing subspace sparsity issue. Leveraging l2,1-norm regularization for feature selection is the major strategy in existing methods, which, however, confronts sparsity limitation and parameter-tuning trouble. To circumvent this problem, employing the l2,0-norm constraint to improve the sparsity of the model has gained more attention recently whereas, optimizing the subspace sparsity constraint is still an unsolved problem, which only can acquire an approximate solution and without convergence proof. To address the above challenges, we innovatively propose a novel subspace sparsity discriminative feature selection (S²DFS) method which leverages a subspace sparsity constraint to avoid tuning parameters. In addition, the trace ratio formulated objective function extremely ensures the discriminability of selected features. Most important, an efficient iterative optimization algorithm is presented to explicitly solve the proposed problem with a closed-form solution and strict convergence proof. To the best of our knowledge, such an optimization algorithm of solving the subspace sparsity issue is first proposed in this article, and a general formulation of the optimization algorithm is provided for improving the extensibility and portability of our method. Extensive experiments conducted on several high-dimensional text and image datasets demonstrate that the proposed method outperforms related state-of-the-art methods in pattern classification and image retrieval tasks.

This article presents a novel discriminative subspace-learning-based unsupervised domain adaptation (DA) method for the gas sensor drift problem. Many existing subspace learning approaches assume that the gas sensor data follow a certain distribution such as Gaussian, which often does not exist in real-world applications. In this article, we address this issue by proposing a novel discriminative subspace learning method for DA with neighborhood preserving (DANP). We introduce two novel terms, including the intraclass graph term and the interclass graph term, to embed the graphs into DA. Besides, most existing methods ignore the influence of the subspace learning on the classifier design. To tackle this issue, we present a novel classifier design method (DANP+) that incorporates the DA ability of the subspace into the learning of the classifier. The weighting function is introduced to assign different weights to different dimensions of the subspace. We have verified the effectiveness of the proposed methods by conducting experiments on two public gas sensor datasets in comparison with the state-of-the-art DA methods.

As an indispensable part of Internet of Vehicles (IoV), unmanned aerial vehicles (UAVs) can be deployed for target positioning and navigation in space-air-ground integrated networks (SAGIN) environment. Maritime target positioning is very important for the safe navigation of ships, hydrographic surveys, and marine resource exploration. Traditional methods typically exploit satellites to locate marine targets in SAGIN environment, and the location accuracy does not satisfy the requirements of modern ocean observation missions. In order to localize marine target, we develop a system architecture in this paper, which contains UAVs integrated with monostatic multiple-input multiple-output (MIMO) radars. The main thrust is to estimate direction-of-arrival (DOA) via MIMO radar. Herein, we consider a general scenario that unknown mutual coupling exist, and a novel sparse reconstruction algorithm is proposed. The mutual coupling matrix (MCM) is adopted with the help of its special structure, we formulate the data model as a sparse representation form. Then two novel matrices, a weighted matrix and a reduced-dimensional matrix are constructed to reduce the computational complexity and enhance the sparsity, respectively. Thereafter, a sparse constraint model is constructed using the concept of optimal weighted subspace fitting (WSF). Finally, DOA estimation of maritime targets can be achieved by reconstructing the support of a block sparse matrix. Based on the DOA estimation results, multiple UAVs are used to cross-locate marine targets multiple times, and an accurate marine target position is achieved in SAGIN environment. Numerical results are carried out, which demonstrates the effectiveness of the proposed DOA estimator, and the multi-UAV cooperative localization system can realize accurate target localization.

Domain adaptation, which transfers the knowledge from label-rich source domain to unlabeled target domains, is a challenging task in machine learning. The prior domain adaptation methods focus on pairwise adaptation assumption with a single source and a single target domain, while little work concerns the scenario of one source domain and multiple target domains. Applying pairwise adaptation methods to this setting may be suboptimal, as they fail to consider the semantic association among multiple target domains. In this work we propose a deep semantic information propagation approach in the novel context of multiple unlabeled target domains and one labeled source domain. Our model aims to learn a unified subspace common for all domains with a heterogeneous graph attention network, where the transductive ability of the graph attention network can conduct semantic propagation of the related samples among multiple domains. In particular, the attention mechanism is applied to optimize the relationships of multiple domain samples for better semantic transfer. Then, the pseudo labels of the target domains predicted by the graph attention network are utilized to learn domain-invariant representations by aligning labeled source centroid and pseudo-labeled target centroid. We test our approach on four challenging public datasets, and it outperforms several popular domain adaptation methods.

We show that the classification performance of graph convolutional networks (GCNs) is related to the alignment between features, graph, and ground truth, which we quantify using a subspace alignment measure (SAM) corresponding to the Frobenius norm of the matrix of pairwise chordal distances between three subspaces associated with features, graph, and ground truth. The proposed measure is based on the principal angles between subspaces and has both spectral and geometrical interpretations. We showcase the relationship between the SAM and the classification performance through the study of limiting cases of GCNs and systematic randomizations of both features and graph structure applied to a constructive example and several examples of citation networks of different origins. The analysis also reveals the relative importance of the graph and features for classification purposes., Comment: Published in IEEE Transactions on Neural Networks and Learning Systems; Date of Publication: 11 January 2021

This paper presents a closed-loop time-varying continuous-time recursive subspace-based prediction method utilizing principle angles rotation. A simple linear mapping can be provided by generalized Poisson moment functionals, which can deal with the time-derivatives problems of input–output Hankel matrices. The parity space employed in fault detection field is adopted instead of using the observable subspace. The system matrices are estimated consistently by the instrumental variable method and principal component analysis, which solves the identification problems of biased results for the system operating in closed-loop with a feedback controller. The system matrices are predicted by the principle angles rotation of the signal subspaces spanned from the extended observability matrices. The effectiveness of the proposed method is illustrated by the numerical simulations and real applications.

Fuzzy K-Means (FKM) clustering is of great importance for analyzing unlabeled data. FKM algorithms assign each data point to multiple clusters with some degree of certainty measured by the membership function. In these methods, the fuzzy membership degree matrix is obtained based on the calculation of the distance between data points in the original space. However, this operation may lead to suboptimal results because of the influence of noises and redundant features. Besides, some FKM clustering methods ignore the importance of the weighting exponent. In this paper, we propose a novel FKM method called Fuzzy K-Means Clustering With Discriminative Embedding. Within this method, we simultaneously conduct dimensionality reduction along with fuzzy membership degree learning. To retain most information in the embedding subspace and improve the robustness of this method, principal component analysis is incorporated into our framework. An iterative optimization algorithm is proposed to solve the model. To validate the efficacy of the proposed method, we perform comprehensive analyses, including convergence behavior, parameter determination and computational complexity. Moreover, we also match a appropriate weighting exponent for each data set. Experimental results on benchmark data sets show that the proposed method is more discriminative and effective for clustering tasks.

The traditional collaborative fuzzy clustering can effectively perform data clustering in distributed peer-to-peer (P2P) networks, which is an impossible task to complete for the centralized clustering methods due to privacy and security requirements or network transmission technology constraints. But it will increase the number of clustering iterations and lead to lower efficiency of the clustering. Moreover, the collaborative mechanism hidden in the iterative process of clustering cannot be well revealed and explained. In this paper, a novel series of transfer collaborative fuzzy clustering algorithms are proposed to solve these issues. In the first basic algorithm, the transfer learning among neighbor nodes vividly expresses the collaborative mechanism and enhances the information collaboration to accelerate the convergence of fuzzy clustering. Meanwhile, neighbor nodes can learn the knowledge from each other to further promote their respective clustering performance. Then, an improved version with the learning-rate-adjustable strategy instead of fixed values is designed to highlight the different influence between neighbor nodes, and the appropriate learning rates between neighbor nodes are achieved to ensure the stable clustering accuracy. Finally, two extended versions with the attribute-weight-entropy regularization technique are presented for the clustering of high dimensional sparse data and the extraction of important subspace features. Experiments show the efficiency of the proposed algorithms compared with the related prototype-based clustering methods.

Hyperspectral images (HSIs) are often contaminated by several types of noise, which significantly limits the accuracy of subsequent applications. Recently, low-rank modeling based on tensor singular value decomposition (T-SVD) has achieved great success in HSI restoration. Most of them use the convex and nonconvex surrogates of the tensor rank, which cannot well approximate the tensor singular values and obtain suboptimal restored results. We suggest a novel HSI restoration model by introducing a fibered rank constrained tensor restoration framework with an embedded plug-and-play (PnP)-based regularization (FRCTR-PnP). More precisely, instead of using the convex and nonconvex surrogates to approximate the fibered rank, the proposed model directly constrains the tensor fibered rank of the solution, leading to a better approximation to the original image. Since exploiting the low-fibered-rankness of HSI is mainly to capture the global structure, we further employ an implicit PnP-based regularization to preserve the image details. Particularly, the above two building blocks are complementary to each other, rather than isolated and uncorrelated. Based on the alternating direction multiplier method (ADMM), we propose an efficient algorithm to tackle the proposed model. For robustness, we develop a three-directional randomized T-SVD (3DRT-SVD), which preserves the intrinsic structure of the clean HSI and removes partial noise by projecting the HSI onto a low-dimensional essential subspace. Extensive experimental results including simulated and real data demonstrate that the proposed method achieves superior performance over compared methods in terms of quantitative evaluation and visual inspection.

Various palmprint recognition methods have been proposed and applied in security, particularly authentication. However, improving the performance of palmprint recognition with insufficient training samples per person is still a challenging task. The undersampling problem limits the application and popularization of palmprint recognition. In this article, by regularly sampling different local regions of the palmprint image, we learn complete and discriminative convolution features by using deep convolutional neural networks (DCNNs). With this powerful palmprint description, a joint constrained least-square regression (JCLSR) framework, which performs representation for each local region of the same palmprint image requiring all regular local regions of the palmprint image to have similar projected target matrices, is presented to exploit the commonality of different patches. The proposed method can well solve the undersampling classification problem in palmprint recognition. Experiments were conducted on the IITD, CASIA, noisy IITD, and PolyU multispectral palmprint databases. It can be seen from the experimental results that the proposed JCLSR consistently outperformed the classical palmprint recognition methods and some subspace learning-based methods for palmprint recognition.

In the context of constant--dimension subspace codes, an important problem is to determine the largest possible size $A_q(n, d; k)$ of codes whose codewords are $k$-subspaces of $\mathbb{F}_q^n$ with minimum subspace distance $d$. Here in order to obtain improved constructions, we investigate several approaches to combine subspace codes. This allow us to present improvements on the lower bounds for constant--dimension subspace codes for many parameters, including $A_q(10, 4; 5)$, $A_q(12, 4; 4)$, $A_q(12, 6, 6)$ and $A_q(16, 4; 4)$., 17 pages; construction for A_(10,4;5) was flawed

In this article, a data-driven design scheme of undetectable false data-injection attacks against cyber-physical systems is proposed first, with the aid of the subspace identification technique. Then, the impacts of undetectable false data-injection attacks are evaluated by solving a constrained optimization problem, with the constraints of undetectability and energy limitation considered. Moreover, the detection of designed data-driven false data-injection attacks is investigated via the coding theory. Finally, the simulations on the model of a flight vehicle are illustrated to verify the effectiveness of the proposed methods.

The ever-increasing spectral resolution of hyperspectral images (HSIs) is often obtained at the cost of a decrease in the signal-to-noise ratio of the measurements, thus calling for effective denoising techniques. HSIs from the real world lie in low-dimensional subspaces and are self-similar. The low dimensionality stems from the high correlation existing among the reflectance vectors, and self-similarity is common in real-world images. In this article, we exploit the above two properties. The low dimensionality is a global property that enables the denoising to be formulated just with respect to the subspace representation coefficients, thus greatly improving the denoising performance and reducing the computational complexity during processing. The self-similarity is exploited via a low-rank tensor factorization of nonlocal similar 3-D patches. The proposed factorization hinges on the optimal shrinkage/thresholding of the singular value decomposition (SVD) singular values of low-rank tensor unfoldings. As a result, the proposed method is user friendly and insensitive to its parameters. Its effectiveness is illustrated in a comparison with state-of-the-art competitors. A MATLAB demo of this work is available at https://github.com/LinaZhuang for the sake of reproducibility.

This letter is on observability of discrete-time LTI systems under unknown piece-wise constant inputs with sufficiently slow, but arbitrary update times. Assuming knowledge of the update times, we characterize the unobservable subspace and show that with sufficiently many measurements in each inter-update interval of the input, the unobservable subspace remains fixed. We explore the implications of the result for privacy in event-triggered control through an illustrative example.

One of the best options for managing energy markets and power system performance optimization is precise Electricity Price Forecasting (EPF). In detecting the prices of energy, some factors are of stochastic behavior which has made an unwieldy task. In this study, carrying out Tensor Canonical Correlation Analysis (TCCA) is the first activity to choose the operative parameters in EPF issues and removing redundant factors with low correlation ranks. The second activity is making use of a Deep Neural Network (DNN) with a novel Stacked Pruning Sparse Denoising Autoencoder (SPSDAE) to decrease the noise of data sets with different sources individually. The third activity is that a novel method which is called the Maximum Separation Subspace (MASES) in Sufficient Dimension Reduction (SDR) with Categorical Response. This method is used to detect the outstanding properties of the input data and removing the uncertainty characteristics. The new multi-modal combined (MMC) method is the final step that needs to be taken to predict the day-ahead (one day in advance) cost of electricity. The proposed method is investigated based on the data that were collected from Australia The combination of various predicting models and suggested methods might develop the function of predicting possibility by supplying divergent types of compactness performance. The combined model comprised of three predicting possibilities called beta circulation fitting, scattered Bayesian learning, core compactness prediction. Predicting data respectively are used as a yardstick to compare the efficiency of forecasted results and consequently assess the proposed method. Results of this study confirmed a decrease of effectual error criterion and accuracy enhancement in EPF field as the advantageous effects of this proposed SDR structure.

A fuzzy compactum in a space X is an upper semi-continuous map f : A → [ 0 , 1 ] from a nonempty compact subspace A ⊂ X to the unit closed interval [ 0 , 1 ] such that { x ∈ A : f ( x ) > 0 } is dense in A. In this paper, we investigate the topological structure of the space K e ( X ) of fuzzy compacta of a separable metric space X with the Hausdorff metric. In particular, we prove that K e ( X ) ≈ l 2 if X is an infinite, locally compact, locally connected, connected and separable metric space.

This article presents a new approach to construct reduced-order models for large nonlinear circuits excited by periodical or quasi-periodical sources. The key feature in the constructed reduced model is that it enables, through solving a much smaller system, tracing the variations in the periodical steady-state response of the original circuit with regards to variations in selected design parameters. The proposed approach is based on the notion of using discrete empirical interpolation to project the nonlinear operator of the circuit equation onto a smaller subspace. In addition, the original system equations are projected onto the subspace spanned by the first few derivatives of the solution with respect to the selected design parameters. Computational savings are made possible through solving a reduced system of equations instead of the full system. Numerical examples are presented to validate the accuracy and efficiency of the proposed approach.

Exploring the data-driven prediction strategy of dielectric constant (e) is attractive for the rational design of polymer dielectrics with targeted property, especially for the design of high e and low loss dielectric energy storage. To accelerate the design and discovery of novel polymer-based dielectric energy storage, the machine learning-based predictor, interval support vector regression with optimized genetic algorithm (OGA-ISVR), is proposed to predict e values, which could improve prediction accuracy and reduce time consumption via splitting the overall data space into subspaces, then adaptively choosing the kernel function and obtaining optimal hyper-parameters by genetic algorithm in each subspace. Here, the developed model is sufficiently trained and tested from the experimentally measured data and density functional theory-based computational data at various frequencies (spanning from 60 Hz to 1015 Hz). The mapping relationships between features and property and influencing factor of e values are identified by this machine learning-based model. Furthermore, compared with common support vector regression method, the proposed model has lower computing overhead and higher prediction accuracy. The proposed model is successfully demonstrated here for the instant property predictions of polymer dielectrics.

Hashing methods have been extensively applied to efficient multimedia data indexing and retrieval on account of the explosion of multimedia data. Cross-modal hashing usually learns binary codes by mapping multi-modal data into a common Hamming space. Most supervised methods utilize relation information like class labels as pairwise similarities of cross-modal data pair to narrow intra-modal and inter-modal gap. In this paper, we propose a novel supervised cross-modal hashing method dubbed Subspace Relation Learning for Cross-modal Hashing (SRLCH), which exploits relation information of labels in semantic space to make similar data from different modalities closer in the low-dimension Hamming subspace. SRLCH preserves the modality relationships, the discrete constraints and nonlinear structures, while admitting a closed-form binary codes solution, which effectively enhances the training efficiency. An iterative alternative optimization algorithm is developed to simultaneously learn both hash functions and unified binary codes. With these binary codes and hash functions, we can index multimedia data and search them in an efficient way. Evaluations in two cross-modal retrieval tasks on several widely-used datasets show that the proposed SRLCH outperforms most cross-modal hashing methods. Theoretical analysis also illustrates reasons for our method’s promotion in subspace relation learning.

This letter investigates the disturbance decoupling problem (DDP) for Boolean control networks (BCNs), and proposes some new perspectives on the solvability of DDP. The DDP of BCNs is generalized in the sense that the output-friendly subspace is a subset of the invariant subspace, based on which, a new criterion is presented to design state feedback DDP controllers for a given output-friendly coordinate frame. The study of an illustrative example shows that additional state feedback DDP controllers can be obtained with respect to those that can be obtained with the method proposed in the existing literature.

This paper proposed a novel Robust Multi-View Continuous Subspace Clustering (RMVCSC) algorithm, which can untangle heavily mixed clusters by optimizing a single continuous objective. The proposed objective uses robust estimators to automatically clip specious inter-cluster connections while maintaining convincing intra-cluster correspondences in the common representation subspace learned from multiple views. The common representation subspace can reveal the underlying cluster structure in data. RMVCSC is optimized in an alternating minimization scheme, in which the clustering result and the common representation subspace are simultaneously optimized. Since different views can describe distinct perspectives of input data, the proposed algorithm has more accurate clustering performance than conventional algorithms by exploring information among multi-view data. In other words, the proposed algorithm optimizes a novel continuous objective in the simultaneously learned common representation subspace across multiple views. By using robust redescending estimators, the proposed algorithm is not prone to stick into bad local minima even with outliers in data. This kind of robust continuous clustering methods has never been used for multi-view clustering before. Moreover, the convergence of the proposed algorithm is theoretically proved, and the experimental results show that the proposed RMVCSC can outperform several very recent proposed algorithms in terms of clustering accuracy.

Recently, many scenarios, such as affective disorders treatment, have sparked rising needs for establishment of personalized emotion recognition (PER) models. Unfortunately, the data sparsity issue violates the basic i.i.d. assumption of supervised learning (i.e., training data and test data are independently and identically distributed). In this paper, we present a semi-supervised joint domain adaption (SSJDA) solution, aiming to inject the hidden domain knowledge from ample labeled data of multiple source individuals into the target subject’s customized model. Specifically, we put forward a novel Progressive Low-Rank Subspace Alignment (PLRSA) approach, which unifies a semi-supervised instance-transfer paradigm and an unsupervised mapping-transfer learning paradigm in a single optimization framework. We leverage the boosting-based TrAdaBoost algorithm and the Transfer Component Analysis (TCA) algorithm for the implementation of instance reweighting and feature matching, respectively. Then we introduce the l 2 , 1 - norm to pass feedback and make the joint learning feasible. The central idea is to progressively minimize the cross-domain distribution discrepancies to finally construct the optimal domain-invariant features. We systematically compare the PLRSA method with five state-of-the-art techniques using two public EEG datasets (DEAP and SEED). Both many-to-one and one-to-one evaluations are performed. The experimental results have confirmed the efficacy of the proposed method.

The recent years have witnessed significant advances in deep video action recognition. However, the performances of deep learning-based video action recognition methods are limited in the case of relatively small samples or no labeled samples. Therefore, trying to use unlabeled video data to generate clustering label is essential for small sample learning and zero sample learning. In this paper, we propose a novel deep video action clustering network, which aims to learn the similarity relationship among the unlabeled video samples, and generate the clustering label for each video sample. Specifically, the proposed method simultaneously learns the spatio-temporal features and subspace representations under a jointly optimized framework. It consists of a 3D U-Net self-representation generator, a video-clip reconstruction discriminator, and a confidence-based feedback mechanism. The 3D U-Net self-representation generator learns the spatio-temporal features of the video clips and produces subspace representation matrix. Then, the similarity graph is constructed based on this subspace representation matrix, and the clustering result is obtained. In the learning procedure, a confidence-based feedback mechanism is designed to feed the high-confidence labels of partial samples back to further guide the subspace structure learning, so that the optimal result can be obtained. During training, the video-clip reconstruction discriminator is introduced to evaluate the reconstructed video clips, which is beneficial for capturing the discriminative spatio-temporal features. Experimental results on a video benchmark dataset demonstrate the effectiveness of the proposed method.

We propose new repair schemes for Reed-Solomon codes that use subspace polynomials and hence generalize previous works in the literature that employ trace polynomials. The Reed-Solomon codes are over $\mathbb {F}_{{q}^{\ell }}$ and have redundancy ${{r}} = {{n}}-{{k}} \geq {{q}}^{{m}}$ , $1\leq {{m}}\leq \ell $ , where ${{n}}$ and ${{k}}$ are the code length and dimension, respectively. In particular, for one erasure, we show that our schemes can achieve optimal repair bandwidths whenever ${{n}}={{q}}^\ell $ and ${{r}} = {{q}}^{{m}}$ , for all $1 \leq {{m}} \leq \ell $ . For two erasures, our schemes use the same bandwidth per erasure as the single erasure schemes, for $\ell /{{m}}$ is a power of ${{q}}$ , and for $\ell ={{q}}^{{a}}$ , ${{m}}={{q}}^{{b}}-1>1$ ( ${{a}} \geq {{b}} \geq 1$ ), and for ${{m}}\geq \ell /2$ when $\ell $ is even and ${{q}}$ is a power of two.

Taking advantage of labeled auxiliary training data whose distribution is similar to the distribution of the gallery, single sample face recognition (SSFR) has achieved encouraging performance. However, in many real-world applications, it is difficult to collect such an auxiliary training dataset, while it may be easier to collect an unlabeled target training dataset whose distribution is similar to the distribution of the gallery and a labeled source training dataset whose distribution may be different to the distribution of the gallery. How can these three datasets be effectively leveraged to handle SSFR? To address this issue, this paper proposes a new method of G allery- S ensitive Single Sample Face Recognition based on D omain A daptation (GS-DA). First, GS-DA employs the method of TSD (targetize the source domain) to construct a common subspace and a targetized source domain. Secondly, it projects each gallery image into the common subspace and obtains the sparse representation of each gallery image in the common subspace. Thirdly, it reconstructs each gallery image from the targetized source domain to estimate the within-class scatter matrix and the between-class scatter matrix of the gallery. Lastly, it learns a discriminant model by maximizing the sum of the traces of the between-class scatter matrix of the gallery and the between-class scatter matrix of the targetized source domain as well as minimizing the sum of the traces of the total scatter matrix of the gallery and the total scatter matrix of the target training data. The experimental results on five datasets illustrate the superiority of GS-DA in leveraging these three datasets for SSFR.

Semi-supervised learning is a new challenge that exploits the information of unlabeled instances. In this paper, we propose a novel Laplacian pair-weight vector projection (LapPVP) algorithm for semi-supervised classification . LapPVP consists of two stages: projection and classification. In the projection stage, LapPVP integrates a within-class scatter, a between-class scatter and a Laplacian regularization together and formulates a pair of optimization problems. The goal of LapPVP is to maximize the between-class scatter and minimize the within-class scatter on the basis of labeled data, and simultaneously maintain the geometric structure of labeled and unlabeled data in the projected subspace . The optimization problems of LapPVP are identical to generalized eigenvalue ones. Thus, it is easy to obtain the optimal solutions to LapPVP. In the classification stage, LapPVP adopts a minimal distance classifier to implement tasks in the projected subspace. The proposed LapPVP can find a better separability using the geometric information embedded in labeled and unlabeled instances and the discriminative information embedded in labeled instance. Experiments on artificial datasets and UCI datasets validate the feasibility and effectiveness of the proposed algorithm.

In this paper, the new closed-loop current harmonics compensation strategy is introduced in the switching-table-based direct torque control (ST-DTC) of dual three-phase permanent magnet synchronous machines (dual 3-ph PMSMs). The harmonic current distortion is the major problem of ST-DTC of multiphase machines and is typically caused by the lack of control in the auxiliary x-y subspace under the vector space decomposition model. The synthetic vectors to obtain average zero voltage in the x-y subspace can be a solution. However, this cannot compensate for current harmonics, for example, due to dead-time and backelectromotive force distortion, which are also mapped into the x-y subspace. Therefore, in this work the x-y voltage references are obtained by the closed control loop that controls x-y currents to zero. To modulate the non-zero x-y voltage reference, the new modulation approach employing voltage vector groups in x-y subspace together with the new switching table is developed and incorporated into ST-DTC. Meanwhile, the analysis of the two available sets of twelve voltage vector groups illustrates a trade-off between linear range in x-y subspace and the DC link voltage utilization rate. Finally, the effectiveness of the proposed ST-DTC strategy is verified through experiments and simulation.

Individual pitch control (IPC) is an effective and widely used strategy to mitigate blade loads in wind turbines. However, conventional IPC fails to cope with blade and actuator faults, and this situation may lead to an emergency shutdown and increased maintenance costs. In this paper, a fault‐tolerant individual pitch control (FTIPC) scheme is developed to accommodate these faults in floating offshore wind turbines (FOWTs), based on a Subspace Predictive Repetitive Control (SPRC) approach. To fulfill this goal, an online subspace identification paradigm is implemented to derive a linear approximation of the FOWT system dynamics. Then, a repetitive control law is formulated to attain load mitigation under operating conditions, both in healthy and faulty conditions. Since the excitation noise used for the online subspace identification may interfere with the nominal power generation of the wind turbine, a novel excitation technique is developed to restrict excitation at specific frequencies. Results show that significant load reductions are achieved by FTIPC, while effectively accommodating blade and actuator faults and while restricting the energy of the persistently exciting control action.

Multi-head attention is appealing for the ability to jointly attend to information from different representation subspaces at different positions. In this work, we propose two approaches to better exploit such diversity for multi-head attention, which are complementary to each other. First, we introduce a disagreement regularization to explicitly encourage the diversity among multiple attention heads. Specifically, we propose three types of disagreement regularization, which respectively encourage the subspace, the attended positions, and the output representation associated with each attention head to be different from other heads. Second, we propose to better capture the diverse information distributed in the extracted partial-representations with the routing-by-agreement algorithm. The routing algorithm iteratively updates the proportion of how much a part (i.e. the distinct information learned from a specific subspace) should be assigned to a whole (i.e. the final output representation), based on the agreement between parts and wholes. Experimental results on the machine translation, sentence encoding and logical inference tasks demonstrate the effectiveness and universality of the proposed approaches, which indicate the necessity of better exploiting the diversity for multi-head attention. While the two strategies individually boost performance, combining them together can further improve the model performance.

The increasing of illegal radiations, which are either artificial or unintentional, has seriously influenced the reliable communication and operation of industrial facilities. In this article, we discuss the simultaneous localization of multiple emitters based on big data monitored by a moving unmanned aerial vehicle. Conventional direct position determination (DPD) method suffers from the non-homogeneity of the observation error and is sensitive to the environment, so we develop the weight DPD methods. First, we consider to strengthen the spectrums obtained at slots with higher signal-to-noise ratio, which is blindly calculated. Thereafter, an improved weight is designed to further enhance the localization accuracy, and it can obtain the asymptotically optimal performance under the general Gaussian noise model, which is proved theoretically. Numerical simulations demonstrate that the proposed weight DPD methods outperform the conventional two-step methods and subspace data fusion DPD method in terms of localization accuracy and resolution.

Symmetry is an important property of human faces. Many researchers exploit symmetry to improve face recognition. In this paper, we explore the symmetry of faces under the framework of PCA (Principal Component Analysis). Unlike previous studies that manipulate facial symmetry by averaging the two halves, we argue that well-estimated symmetrical faces lie in a low-dimensional subspace. Inspired by this, a PCA-like model, referred to as Mirror PCA, is proposed to extract low-rank symmetrical features of faces, as well as their principal directions. Although the optimization problem is non-convex, a suboptimal solution could be found under existing optimization frameworks such as ADMM, PGD, etc. A number of experiments are conducted to verify the effectiveness and robustness of our method.

Recently, robust-norm distance related principal component analysis (PCA) for feature extraction has been shown to be very effective for image analysis, which considers either minimization of reconstruction error or maximization of data variance in low-dimensional subspace . However, both of them are important for feature extraction. Furthermore, most of existing methods cannot obtain satisfactory results due to the utilization of inflexible robust norm for distance metric. To address these problems, this paper proposes a novel robust PCA formulation called Double L 2,p-norm based PCA (DLPCA) for feature extraction, in which the minimization of reconstruction error and the maximization of variance are simultaneously taken into account in a unified framework. In the reconstruction error function, we target to learn a latent subspace to bridge the relationship between the transformed features and the original features. To guarantee the objective to be insensitive to outliers, we take L2,p-norm as the distance metric for both reconstruction error and data variance. These characteristics make our method more applicable for feature extraction. We present an effective iterative algorithm to obtain the solution of this challenging work, and conduct theoretical analysis on the convergence of the algorithm. The experimental results on several databases show the effectiveness of our model.

In recent years, the domain adaption has received wide attention from machine learning communities because of differences in data distribution or the lack of training data in a practical machine learning task. In this work, we propose a Pairwise Attention Network (PAN for short) for addressing cross-domain image recognition task. In this model, different local features and the global-feature are concatenated to obtain different attention estimators, and then they are combined to get the attention map. In this way, we can focus on the important parts of an image, and ignore the irrelative regions. Moreover, attention consistency is also embedded in PAN to make sure consistent interest regions in the same class. Besides, to improve the feature discrimination, an embedding discriminative subspace is learned where it maps positive sample pairs aligned in a hypersphere and negative sample pairs separated. Extensive experimental results on the MNIST-USPS, office, and Visda-2017 datasets demonstrate that PAN can outperform state-of-the-art methods in terms of average accuracy.

Deep neural networks are demonstrated to be vulnerable to adversarial examples. In this paper, starting from the robustness analysis about the model ensemble, we propose a novel type of defense method named “Self-Orthogonal Randomization Super-network” (SORS). More specifically, we think the main robustness benefit from the model ensemble comes from two aspects: smaller adversarial subspace and gradient orthogonality. However, the naive model ensemble has two fundamental limitations: 1) Though ensembling more models will introduce more robustness, training too many models is infeasible and resource-consuming. 2) Since these models are usually trained independently, the gradient orthogonality among them is often partial and weak. Motivated by this, we propose to train one single super-network that consists of the exponential number of sub-networks, and explicitly constrain the gradient of different sub-networks with respect to the same input to be orthogonal. In the inference stage, at each forward pass, one sub-network will be randomly sampled. Through extensive experiments, we demonstrate that the proposed method can achieve significantly better robustness than the vanilla single model baseline and the naive model ensemble baseline. Moreover, this new type of defense strategy is also complementary to other types of defense methods and achieves state-of-the-art performance.

In this work, a data-driven technique is proposed for efficient design exploration and optimisation of the Kaplan turbine. To avoid the curse of dimensionality, the proposed approach commences with the extraction of latent features of a parametric design space, which form a lower-dimensional subspace accumulating maximum geometric variability of designs. Afterwards, this subspace is exploited for the construction of a Gaussian Process-based surrogate model using an adaptive training strategy to infer the relative-tangential velocities at the leading and trailing edges of the turbine. The training strategy is structured on a high-fidelity sampling approach to ensure a notable prediction accuracy with a few training samples. After training, the surrogate model is integrated with an optimiser to explore the subspace for an optimal design and to determine the sensitivity of design parameters. The results showed that the optimal design generated with the proposed method increases the efficiency of the initial design from 56.98% to 90.73% at a significantly low computational cost. Finally, the convergence performance is verified with different experimentation and its accuracy to extract latent features and to predict the relative-tangential velocity is demonstrated via a comparative study in which different state-of-the-art approaches are compared with the proposed approach.

Recently, collaborative representation classification (CRC) has attracted extensive interest for hyperspectral images (HSIs) classification. However, for collaborative representation with Tikhonov (CRT), a testing sample is collaboratively represented by training samples from all the classes, which may result in high computational cost. In this article, we select the first $k$ class training samples that are nearest to the testing sample for representation, namely, $k$ -nearest class CRT (KNCCRT) algorithm. In order to improve the performance of KNCCRT for HSI classification, the idea of random subspace-based KNCCRT ensemble framework is proposed. KNCCRT is adopted as base classifier and random subspace (RS) contributes to diversity by selecting feature randomly. Moreover, to further increase the classification accuracy, shape-adaptive (SA) neighborhood constraint is utilized in RS ensemble framework to incorporate spatial information. Experimental results on three real hyperspectral data sets demonstrate the effectiveness of the proposed methods for HSI classification. The combination of KNCCRT and RS framework provides a reliable accuracy for HSI classification.

A novel alternate evolution of sub-populations based on tensor decomposition called, TASE is proposed, for solving multi- and many-objective optimization problems with large-scale decision variables in this work. Tensor canonical polyadic (CP) decomposition is introduced for the first time to divide the heterogeneous variables of higher-dimensional decision space into several lower-dimensional sub-components. Furthermore, these sub-populations are optimized alternatively to search for improved solutions in their respective lower-dimensional decision subspace . Finally, a cross-population matching scheme is designed to reconstruct the whole population accurately. The experiments use some largescale multi- and many-objective problems with 2–6 objectives and 1000–5000 variables. The proposed algorithm is compared with other state-of-the-art algorithms, and the experimental results indicate that it can solve some problems that other well-known large-scale optimization algorithms cannot, as well as outperforming other algorithms in terms of solution quality and convergence rate.

For imbalanced credit scoring, the most common solution is to balance the class distribution of the training set with a resampling method , and then train a classification model and classify the customer samples in the test set. However, it is still difficult to select the most appropriate resampling methods and classification models, and the optimal combinations of them have not been identified. Therefore, this study proposes a new benchmark models comparison framework for imbalanced credit scoring. In the framework, we introduce the index of balanced accuracy and four other evaluation measures, experimentally compare the performance of 10 benchmark resampling methods and nine benchmark classification models respectively on six credit scoring data sets, and analyze the optimal combinations of them. The experimental result shows: (1) as for benchmark resampling methods, random under-sampling (a traditional resampling method) and synthetic minority over-sampling technique combined with Wilson’s edited nearest neighbor (an intelligent resampling method) present the best performance; (2) as for benchmark classification models, logistic regression (a single classification model) and adaptive boosting (an ensemble classification model) present the best performance; (3) as for optimal combinations, random under-sampling combined with random subspace (an ensemble classification model) can obtain the most satisfactory credit scoring performance.

This article presents a reliable islanding detection scheme for distributed generation (DG) to minimize the nondetection zone using a pattern-recognition method. A hybrid time–frequency signal decomposition along with a machine learning processes the voltage signal retrieved at the DG to make final decision. Amalgamation of time-varying filter and time domain decomposition obtains a modified intrinsic mode functions (MIMF), which enhances the time–frequency resolution of nonstationary signals. Moreover, the adaptive nature of the proposed hybrid signal decomposition makes it more advisable over other decomposition techniques to frame the input feature vector. Further, the random subspace ensemble framework based on ensemble k-nearest neighbor classifier is used among different machine learning techniques to identify the islanding condition by applying the feature vector generated using MIMF. The proposed scheme is thoroughly verified on two standard test systems for identifying the typical islanding condition of zero power mismatch and the proposed scheme discriminates the islanding from large scale disturbances such as capacitor switching and faults. The performance of the scheme is assessed through reliability analysis and it is also compared to other machine learning techniques.

In the underwater waveguide, the conventional adaptive subspace detector (ASD), derived by using the generalized likelihood ratio test (GLRT) theory, suffers from a significant degradation in detection performance when the samplings of training data are deficient. This paper proposes a dimension-reduced approach to alleviate this problem. The dimension reduction includes two steps: firstly, the full array is divided into several subarrays; secondly, the test data and the training data at each subarray are transformed into the modal domain from the hydrophone domain. Then the modal-domain test data and training data at each subarray are processed to formulate the subarray statistic by using the GLRT theory. The final test statistic of the dimension-reduced ASD (DR-ASD) is obtained by summing all the subarray statistics. After the dimension reduction, the unknown parameters can be estimated more accurately so the DR-ASD achieves a better detection performance than the ASD. In order to achieve the optimal detection performance, the processing gain of the DR-ASD is deduced to choose a proper number of subarrays. Simulation experiments verify the improved detection performance of the DR-ASD compared with the ASD.

The traditional direct torque control (DTC) method will produce a larger harmonic current in the x, y subspace when applied to a dual three-phase permanent magnet synchronous machine (DTP-PMSM) because the voltage vector used for DTC is not equal to zero in the x, y harmonic subspace. To mitigate this problem, in this manuscript, a model predictive direct torque control (MPDTC) method is proposed to eliminate the harmonic current in DTP-PMSM. The spatial distribution characteristic DTP-PMSM voltage vector is analyzed; then, the table of vector group can be obtained according to DTC, and each vector group can be combined to obtain the zero-voltage vector in the x, y subspace. According to the cost function, MPDTC selects the vector group and obtains the optimal vector sequence combination to eliminate the harmonic current in the x, y subspace. Furthermore, the MPDTC achieves closed-loop control of harmonic currents in the x, y subspace. The MPDTC can also eliminate harmonic currents caused by other factors. The simulation results show that the method of MPDTC can effectively suppress the harmonic current of DTP-PMSM.

Hashing methods have sparked a great revolution on large-scale cross-media search due to its effectiveness and efficiency. Most existing approaches learn unified hash representation in a common Hamming space to represent all multimodal data. However, the unified hash codes may not characterize the cross-modal data discriminatively, because the data may vary greatly due to its different dimensionalities, physical properties, and statistical information. In addition, most existing supervised cross-modal algorithms preserve the similarity relationship by constructing an n × n pairwise similarity matrix, which requires a large amount of calculation and loses the category information. To mitigate these issues, a novel cross-media hashing approach is proposed in this article, dubbed label flexible matrix factorization hashing (LFMH). Specifically, LFMH jointly learns the modality-specific latent subspace with similar semantic by the flexible matrix factorization. In addition, LFMH guides the hash learning by utilizing the semantic labels directly instead of the large n × n pairwise similarity matrix. LFMH transforms the heterogeneous data into modality-specific latent semantic representation. Therefore, we can obtain the hash codes by quantifying the representations, and the learned hash codes are consistent with the supervised labels of multimodal data. Then, we can obtain the similar binary codes of the corresponding modality, and the binary codes can characterize such samples flexibly. Accordingly, the derived hash codes have more discriminative power for single-modal and cross-modal retrieval tasks. Extensive experiments on eight different databases demonstrate that our model outperforms some competitive approaches.

The region covariance descriptor (RCD), which is known as a symmetric positive definite (SPD) matrix, is commonly used in image representation. As SPD manifolds have a non-Euclidean geometry, Euclidean machine learning methods are not directly applicable to them. In this work, an improved covariance descriptor called the hybrid region covariance descriptor (HRCD) is proposed. The HRCD incorporates the mean feature information into the RCD to improve the latter’s discriminative performance. To address the non-Euclidean properties of SPD manifolds, this study also proposes an algorithm called the Hilbert-Schmidt independence criterion subspace learning (HSIC-SL) for SPD manifolds. The HSIC-SL algorithm is aimed at improving classification accuracy. This algorithm is a kernel function that embeds SPD matrices into the reproducing kernel Hilbert space and further maps them to a linear space. To make the mapping consider the correlation between SPD matrices and linear projection, this method introduces global HSIC maximization to the model. The proposed method is compared with existing methods and is proved to be highly accurate and valid by classification experiments on the HRCD and HSIC-SL using the COIL-20, ETH-80, QMUL, face data FERET, and Brodatz datasets.

Feature extraction has been widely studied to find informative latent features and reduce the dimensionality of data. In particular, due to the difficulty in obtaining labeled data, unsupervised feature extraction has received much attention in data mining. However, widely used unsupervised feature extraction methods require side information about data or rigid assumptions on the latent feature space. Furthermore, most feature extraction methods require predefined dimensionality of the latent feature space,which should be manually tuned as a hyperparameter. In this article, we propose a new unsupervised feature extraction method called Unsupervised Subspace Extractor ( USE ), which does not require any side information and rigid assumptions on data. Furthermore, USE can find a subspace generated by a nonlinear combination of the input feature and automatically determine the optimal dimensionality of the subspace for the given nonlinear combination. The feature extraction process of USE is well justified mathematically, and we also empirically demonstrate the effectiveness of USE for several benchmark datasets.

Microexpression recognition has been widely favored by researchers due to its many potential applications, such as business negotiation and lie detection. Cross-database microexpression recognition is more challenging and attractive than normal microexpression recognition because the training and testing samples come from different databases. The ensuing challenge is that the feature distributions between training and testing samples differ too much. As a result, the performance of current well-performing microexpression recognition methods often fails to achieve the desired effect. In this paper, we overcome this problem by introducing Subspace Learning and Joint Distribution Adaptation (SLJDA) by projecting the source and target domains into the subspace and later reducing the distance between them and then minimizing the distance between the marginal and conditional probability distributions of the data between the source domain and the target domain. To evaluate its performance, a large number of cross-database experiments are performed in the SMIC database and CASMEII database. The experimental results show the superiority of the method compared with existing microexpression recognition methods.