Search

• Deep metric learning technique is first introduced into the multi-spectral face recognition task. • The spectrum-aware embedding loss takes both the spectrum and class label information into consideration. • The multi-spectral discriminant correlation loss fully exploits the useful correlation information in multi-spectral images. • The proposed approach significantly outperforms state-of-the-art multi-spectral face recognition methods. One primary challenge of face recognition is that the performance is seriously affected by varying illumination. Multi-spectral imaging can capture face images in the visible spectrum and beyond, which is deemed to be an effective technology in response to this challenge. For current multi-spectral imaging-based face recognition methods, how to fully explore the discriminant and correlation features from both the intra-spectrum and inter-spectrum aspects with only a limited number of multi-spectral samples for model training has not been well studied. To address this problem, in this paper, we propose a novel face recognition approach named Spectrum-aware Discriminative Deep Learning (SDDL). To take full advantage of the multi-spectral training samples, we build a discriminative multi-spectral network (DMN) and take face sample pairs as the input of the network. By jointly considering the spectrum and the class label information, SDDL trains the network for projecting samples pairs into a discriminant feature subspace, on which the intrinsic relationship including the intra- and inter-spectrum discrimination and the inter-spectrum correlation among face samples is well discovered. The proposed approach is evaluated on three widely used datasets HK PolyU, CMU, and UWA. Extensive experimental results demonstrate the superiority of SDDL over state-of-the-art competing methods. [ABSTRACT FROM AUTHOR]

• We propose a Modality-Specific and Shared Generative Adversarial Network approach. • The modality-specific and modality-shared features are jointly explored and leveraged. • The inter-modal invariance and the inter- and intra-modal discrimination is well modeled. • Superiority of our approach is demonstrated on multiple benchmark multi-modal datasets. Cross-modal retrieval aims to realize accurate and flexible retrieval across different modalities of data, e.g., image and text, which has achieved significant progress in recent years, especially since generative adversarial networks (GAN) were used. However, there still exists much room for improvement. How to jointly extract and utilize both the modality-specific (complementarity) and modality-shared (correlation) features effectively has not been well studied. In this paper, we propose an approach named Modality-Specific and Shared Generative Adversarial Network (MS2GAN) for cross-modal retrieval. The network architecture consists of two sub-networks that aim to learn modality-specific features for each modality, followed by a common sub-network that aims to learn the modality-shared features for each modality. Network training is guided by the adversarial scheme between the generative and discriminative models. The generative model learns to predict the semantic labels of features, model the inter- and intra-modal similarity with label information, and ensure the difference between the modality-specific and modality-shared features, while the discriminative model learns to classify the modality of features. The learned modality-specific and shared feature representations are jointly used for retrieval. Experiments on three widely used benchmark multi-modal datasets demonstrate that MS2GAN can outperform state-of-the-art related works. [ABSTRACT FROM AUTHOR]

Semantic segmentation networks usually utilize special pyramid structure after encoder or combine low-level and high-level feature maps in decoder to capture multi-scale context information, which we term them feature combination and feature connection, respectively. However, both such frameworks are less valid with the fixed geometric structure or unsuitable interim. In this paper, we advocate Dynamic Attention Network (DAN) to solve these problems. First, we design a Deformable Attention Pyramid (DAP) module to perform a self-adjustable descriptor of high-level output, which utilizes deformable function to model geometric transformation. With DAP, semantic information can be captured effectively. Second, we propose a Fusing Attention Interim (FAI) module to guide the back-propagation of long-short range information in each level of decoder. We evaluate DAN on the challenging PASCAL VOC 2012 and Cityscapes segmentation benchmarks and find that it achieves state-of-the-art results without post-processing. Our observation can be concluded that the flexible structure that possesses dynamic attention mechanism is beneficial to learn multi-scale context information. [ABSTRACT FROM AUTHOR]

Domain adaptation (DA) aims to reduce knowledge gap between domains and improve the prediction ability of models in the target domain. However, the representations learned from feature extraction network often contain redundant information, which is harmful to domain alignment. In addition, many methods only focus on either single-source DA task or multi-source DA task, which limits their real-world applications. In this paper, we propose a simple but effective method called Category and Domain Features Augmentation (CDFA), which consists of two components: Contrastive Classifier Network (CCN) and Domain-specific Learning Network (DSLN). CDFA can remove the specific representation at the feature extraction stage to alleviate transfer difficulty, where CCN is used to increase the probabilistic output of samples and avoid misclassification, and DSLN facilitates the separation of redundant information from all representations by learning domain-specific representations. Empirical evaluations on several cross-domain benchmarks under single-source and multi-source DA scenarios illustrate the competitive performance of CDFA with respect to the state-of-the-art. [Display omitted] • The effects of the classification layer on the sample of decision boundaries is emphasized. • The correlation and independence of all representations with domain-specific representations is investigated. • Removal of redundant information by separating domain-specific representations. • A novel method based on domain separation category and domain feature alignment is proposed. • Successful application in single-source and multi-source domain adaptation scenarios. [ABSTRACT FROM AUTHOR]

• A progressive GAN generation framework based on GANs is proposed to generate highresolution ground-view panorama images solely from low-resolution aerial images. • A novel cross-stage attention module is proposed to bridge adjacent generation stages of the progressive generation process so that the quality of synthesized panorama image could be continually improved. • A novel orientation-aware data augmentation strategy is proposed to utilize geometric relation between aerial and segmentation images for model training. • The proposed model establishes new state-of-the-art results for the task of cross-view panorama scene image synthesis in two scenarios: suburb area and urban area. Despite the significant progress of conditional image generation, it remains difficult to synthesize a ground-view panorama image from a top-view aerial image. Among the core challenges are the vast differences in image appearance and resolution between aerial images and panorama images, and the limited aside information available for top-to-ground viewpoint transformation. To address these challenges, we propose a new Progressive Attention Generative Adversarial Network (PAGAN) with two novel components: a multistage progressive generation framework and a cross-stage attention module. In the first stage, an aerial image is fed into a U-Net-like network to generate one local region of the panorama image and its corresponding segmentation map. Then, the synthetic panorama image region is extended and refined through the following generation stages with our proposed cross-stage attention module that passes semantic information forward stage-by-stage. In each of the successive generation stages, the synthetic panorama image and segmentation map are separately fed into an image discriminator and a segmentation discriminator to compute both later real and fake, as well as feature alignment score maps for discrimination. The model is trained with a novel orientation-aware data augmentation strategy based on the geometric relation between aerial and panorama images. Extensive experimental results on two cross-view datasets show that PAGAN generates high-quality panorama images with more convincing details than state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Recently, actor-critic structure based neural networks are widely used in many reinforcement learning tasks. It consists of two main parts: (i) an actor module which outputs the probability distribution of action, and (ii) a critic module which outputs the predicted value based on the current environment. Actor-critic structure based networks usually need expert demonstration to provide an appropriate pre-training for the actor module, but the demonstration data is often hard or even impossible to obtain. And most of them, such as those used in the maze and robot control tasks, suffer from a lack of proper pre-training and unstable error propagation from the critic module to the actor module, which would result in poor and unstable performance. Therefore, a specially designed module which is called relatively optimal historical information learning (ROHI) is proposed. The proposed ROHI module can record the historical explored information and obtain the relatively optimal information through a customized merging algorithm. Then, the relatively optimal historical information is used to assist in training the actor module during the main learning process. We introduce two complex experimental environments, including the complex maze problem and flipping game, to evaluate the effectiveness of the proposed module. The experimental results demonstrate that the extended models with ROHI can significantly improve the success rate of the original actor-critic structure based models and slightly decrease the number of iteration required to reach the stable phase of value-based networks. [ABSTRACT FROM AUTHOR]

Deformable Convolutional Networks (DCNs) are proposed to solve the inherent limited geometric transformation in CNNs, showing outstanding performance on sophisticated computer vision tasks. Though they can rule out irrelevant image content and focus on region of interest to some degree, the adaptive learning of the deformation is still limited. In this paper, we delve it from the aspects of deformable modules and deformable organizations to extend the scope of deformation ability. Concretely, on the one hand, we reformulate the deformable convolution and RoIpooling by reconsidering spatial-wise attention, channel-wise attention and spatial-channel interdependency, to improve the single convolution's ability to focus on pertinent image contents. On the other hand, an empirical study is conducted on various and general arrangements of deformable convolutions (e.g., connection type) in DCNs. Especially on semantic segmentation, the study yields significant findings for a proper combination of deformable convolutions. To verify the effectiveness and superiority of our proposed deformable modules, we also provide extensive ablation study for them and compare them with other previous versions. With the proposed contribution, our refined Deformable ConvNets achieve state-of-the-art performance on two semantic segmentation benchmarks (PASCAL VOC 2012 and Cityscapes) and an object detection benchmark (MS COCO). [ABSTRACT FROM AUTHOR]

• A tensor dictionary learning method is developed for unsupervised visual domain adaptation. • Disentangled feature is obtained by separating domain factor and class factor. • The model can address the small-sample-size issue and ensure out-of-sample generalization. • The proposed method is compared to mainstream shallow and deep learning methods. Unsupervised domain adaptation aims at learning a classification model robust to data distribution shift between a labeled source domain and an unlabeled target domain. Most existing approaches have overlooked the multi-dimensional nature of visual data, building classification models in vector space. Meanwhile, the issue of limited training samples is rarely considered by previous methods, yet it is ubiquitous in practical visual applications. In this paper, we develop a structured discriminative tensor dictionary learning method (SDTDL), which enables domain matching in tensor space. SDTDL produces disentangled and transferable representations by explicitly separating domain-specific factor and class-specific factor in data. Classification is achieved based on sample reconstruction fidelity and distribution alignment, which is seamlessly integrated into tensor dictionary learning. We evaluate SDTDL on cross-domain object and digit recognition tasks, paying special attention to the scenarios of limited training samples and test beyond training sample set. Experimental results show that our method outperforms existing mainstream shallow approaches and representative deep learning methods by a significant margin. [ABSTRACT FROM AUTHOR]

Existing Universal Domain Adaptation (UniDA) approaches can handle various domain adaptation (DA) tasks, which need no prior information about the category overlap across target and source domains. However, traditional UniDA scenario cannot fully cover every DA scenario, e.g. , Multi-Source DA is absent. Therefore, aiming to simultaneously handle more DA scenarios in nature, we propose the More Universal Domain Adaptation (MUniDA) task. There are three challenges in MUniDA: (i) Category shift between source and target domains; (ii) Domain shift, especially the domain shift among multiple modalities in the source, which is ignored by the current UniDA approaches; (iii) How to recognize common categories across domains? We propose a more universally applicable DA approach that can tackle above challenges without any modification called A ttention Cycle -consistent U niversal N etwork (A-CycleUN). We show through extensive experiments on several benchmarks that A-CycleUN works stably and outperforms baselines across different MUniDA settings. • We introduce a more practical setting called More Universal Domain Adaptation. • A novel end-to-end framework Attention Cycle-consistent Universal Network is proposed. • The proposed method can achieve state-of-the-art performance under the new setting. [ABSTRACT FROM AUTHOR]

Existing regression based tracking methods built on correlation filter model or convolution model do not take both accuracy and robustness into account at the same time. In this paper, we propose a dual-regression framework comprising a discriminative fully convolutional module and a fine-grained correlation filter component for visual tracking. The convolutional module trained in a classification manner with hard negative mining ensures the discriminative ability of the proposed tracker, which facilitates the handling of several challenging problems, such as drastic deformation, distractors, and complicated backgrounds. The correlation filter component built on the shallow features with fine-grained features enables accurate localization. By fusing these two branches in a coarse-to-fine manner, the proposed dual-regression tracking framework achieves a robust and accurate tracking performance. Extensive experiments on the OTB2013, OTB2015, and VOT2015 datasets demonstrate that the proposed algorithm performs favorably against the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

This work focuses on unsupervised visual domain adaptation which is still challenging in visual recognition. Most of the attention has been dedicated to seeking the domain-invariant features of cross-domain data, but they ignores the valuable discriminative information in the source domain. In this paper, we propose a Discriminative Dictionary Evolution (DDE) approach to seek discriminative features robust to domain shift. Specifically, DDE gradually adapts a discriminative dictionary learned from the source domain to the target domain through a dictionary evolving procedure, in which self-selected atoms of the dictionary are updated with ℓ 2 , 1 -norm-based regularization. DDE produces domain-invariant representations for cross-domain visual recognition meanwhile promotes the discriminativeness of the dictionary. Empirical results on real-world data sets demonstrate the advantages of the proposed approach over existing competitive methods. [ABSTRACT FROM AUTHOR]

Point cloud based 3D objection plays a crucial role in real-world applications, such as autonomous driving. In this paper, we propose the Multi-view Semantic Learning Network (MVSLN) for 3D object detection, an approach considering the feature discrimination for LIDAR point cloud. Since the discrete and disordered nature of point cloud, most existing methods ignore the low-level information and focus more on the spatial details of point cloud. To capture the discriminative feature of objects, our MVSLN takes advantages of both spatial and low-level details to further exploit semantic information. Specifically, the Multiple Views Generator (MVG) module in our approach observes the scene from four views by projecting the 3D point cloud to planes with specific angles, which preserves much more low-level features, e.g., texture and edge. To correct the deviation brought by different projection angles, the Spatial Recalibration Fusion (SRF) operation in our approach adjusts the locations of features of these four views, enabling the interaction between different projections. Then the recalibrated features of SRF are sent to the developed 3D Region Proposal Network (RPN) to detect objects. The experimental results on challenging KITTI benchmark verify that our approach achieves a promising performance and outperforms state-of-the-art methods. Furthermore, the discriminative feature extractor brought by exploiting the conspicuous semantic information, leads to encouraging results in the hard-level difficulty of both BEV and 3D object detection tasks, without any help of camera image. [ABSTRACT FROM AUTHOR]

Sparse additive models have shown competitive performance for high-dimensional variable selection and prediction due to their representation flexibility and interpretability. Despite their theoretical properties have been studied extensively, few works have addressed the robustness for the sparse additive models. In this paper, we employ the robust average top-k (AT k) loss as classification error measure and propose a new sparse algorithm, named AT k group sparse additive machine (AT k -GSAM). Besides the robust concern, the AT k -GSAM has well adaptivity by integrating the data dependent hypothesis space and group sparse regularizer together. Generalization error bound is established by the concentration estimate with empirical covering numbers. In particular, our error analysis shows that AT k -GSAM can achieve the learning rate O (n − 1 / 2) under appropriate conditions. We further analyze the robustness of AT k -GSAM via a sample-weighted procedure interpretation, and the theoretical guarantees on grouped variable selection. Experimental evaluations on both simulated and benchmark datasets validate the effectiveness and robustness of the new algorithm. [ABSTRACT FROM AUTHOR]

Highlights • We extract view-independent features to remove the view discrepancy. • We integrate discriminant regularization to learn a discriminant subspace. • We extend single-view local geometry preservation to multi-view scenario. • We integrate local consistency regularization to learn a structured subspace. Abstract Cross-view classification that means to classify samples from heterogeneous views is a significant yet challenging problem in computer vision. An effective solution to this problem is the multi-view subspace learning (MvSL), which intends to find a common subspace for multi-view data. Although great progress has been made, existing methods usually fail to find a suitable subspace when multi-view data lies on nonlinear manifolds, thus leading to performance deterioration. To circumvent this drawback, we propose Multi-view Common Component Discriminant Analysis (MvCCDA) to handle view discrepancy, discriminability and nonlinearity in a joint manner. Specifically, our MvCCDA incorporates supervised information and local geometric information into the common component extraction process to learn a discriminant common subspace and to discover the nonlinear structure embedded in multi-view data. Optimization and complexity analysis of MvCCDA are also presented for completeness. Our MvCCDA is competitive with the state-of-the-art MvSL based methods on four benchmark datasets, demonstrating its superiority. [ABSTRACT FROM AUTHOR]

Highlights • "Observers" greatly increased grooming toward their defeated partners. • Fos expressions were elevated in some brain structures. • OT and GABA neurons were activated in the PVN and ACC, respectively. • Consolation was blocked by an OT or a GABA A receptor antagonist within the ACC. Abstract Consolation, which entails comforting contact directed toward a distressed party, is a common empathetic response in humans and other species with advanced cognition. Here, using the social defeat paradigm, we provide empirical evidence that highly social and monogamous mandarin voles (Microtus mandarinus) increased grooming toward a socially defeated partner but not toward a partner who underwent only separation. This selective behavioral response existed in both males and females. Accompanied with these behavioral changes, c-Fos expression was elevated in many of the brain regions relevant for emotional processing, such as the anterior cingulate cortex (ACC), bed nucleus of the stria terminalis, paraventricular nucleus (PVN), basal/basolateral and central nucleus of the amygdala, and lateral habenular nucleus in both sexes; in the medial preoptic area, the increase in c-Fos expression was found only in females, whereas in the medial nucleus of the amygdala, this increase was found only in males. In particular, the GAD67/c-Fos and oxytocin (OT)/c-Fos colocalization rates were elevated in the ACC and PVN, indicating selective activation of GABA and OT neurons in these regions. The "stressed" pairs matched their anxiety-like behaviors in the open-field test, and their plasma corticosterone levels correlated well with each other, suggesting an empathy-based mechanism. This partner-directed grooming was blocked by pretreatment with an OT receptor antagonist or a GABA A receptor antagonist in the ACC but not by a V1a subtype vasopressin receptor antagonist. We conclude that consolation behavior can be elicited by the social defeat paradigm in mandarin voles, and this behavior may be involved in a coordinated network of emotion-related brain structures, which differs slightly between the sexes. We also found that the endogenous OT and the GABA systems within the ACC are essential for consolation behavior in mandarin voles. [ABSTRACT FROM AUTHOR]

Manifold learning aims to discover the low dimensional space where the input high dimensional data are embedded by preserving the geometric structure. Unfortunately, almost all the existing manifold learning methods were proposed under single view scenario, and they cannot be straightforwardly applied to multiple feature sets. Although concatenating multiple views into a single feature provides a plausible solution, it remains a question on how to better explore the independence and interdependence of different views while conducting manifold learning. In this paper, we propose a multi-view manifold learning with locality alignment (MVML-LA) framework to learn a common yet discriminative low-dimensional latent space that contain sufficient information of original inputs. Both supervised algorithm (S-MVML-LA) and unsupervised algorithm (U-MVML-LA) are developed. Experiments on benchmark real-world datasets demonstrate the superiority of our proposed S-MVML-LA and U-MVML-LA over existing state-of-the-art methods. [ABSTRACT FROM AUTHOR]

• MCGCN for the first time builds cross-modal graph and jointly learns modality-specific and modality-shared features for semi-supervised cross-modal hashing. • MCGCN provides a three-channel network architecture, including two modality-specific channels and a cross-modal channel to model cross-modal graph with heterogeneous image and text features. • To effectively reduce the modality gap, network training is guided by adversarial scheme. • MCGCN obtains state-of-the-art semi-supervised cross-modal hashing performance. Cross-modal hashing maps heterogeneous multimedia data into Hamming space for retrieving relevant samples across modalities, which has received great research interests due to its rapid retrieval and low storage cost. In real-world applications, due to high manual annotation cost of multi-media data, we can only make use of limited number of labeled data with rich unlabeled data. In recent years, several semi-supervised cross-modal hashing (SCH) methods have been presented. However, how to fully explore and jointly utilize the modality-specific (complementarity) and modality-shared (correlation) information for retrieval has not been well studied for existing SCH works. In this paper, we propose a novel SCH approach named Modality-specific and Cross-modal Graph Convolutional Networks (MCGCN). The network architecture contains two modality-specific channels and a cross-modal channel to learn modality-specific and shared representations for each modality, respectively. Graph convolutional network (GCN) is leveraged in these three channels to explore intra-modal and inter-modal similarity, and perform semantic information propagation from labeled data to unlabeled data. Modality-specific and shared representations for each modality are fused with attention scheme. To further reduce the modality gap, a discriminative model is designed, learning to classify the modality of representations, and network training is guided by adversarial scheme. Experiments on two widely used multi-modal datasets demonstrate MCGCN outperforms state-of-the-art semi-supervised/supervised cross-modal hashing methods. [ABSTRACT FROM AUTHOR]

This paper explores recently proposed Universal Multi-source Domain Adaptation (UniMDA) task. UniMDA task is different from existing domain adaptation (DA) tasks, e.g. , Multi-source DA, Closed set DA or Universal DA. UniMDA not only handles multi-source issue, but also needs no prior knowledge about the overlap between the target and source label sets. UniMDA task has three challenges: (i) Domain shift issue among the multiple source domains. (ii) Domain shift issue between target and source domains. (iii) Category shift between the target and each source. Towards tackling the challenges, we formulate a universal multi-source adaptation network termed as M ulti- S ource D ual C ontrastive N etwork (MSDCN), including a transferability rule and contrastive module. In addition, to handle multi-source scenario, we propose (1) pairwise similarities maximization over the examples from multiple source domains, and (2) alternative optimization strategy for training the ensemble of multiple source classifiers end-to-end. The proposed method can handle UniMDA scenario generally, where the label set in each source domain may be different from the target domain, while maintaining the complexity of the method in different DA scenarios. Experiments are conducted on several real-world multi-source benchmarks. The results show that MSDCN could work stably, and exceed the state-of-the-art performance against existing domain adaptation algorithms. • This is an early work that explores Universal Multi-Source Domain Adaptation (UniMDA) setting. • The proposed method can tackle domain and category shift issues across domains. • Experiments show that proposed method outperforms state-of-the-art methods about +4.26%. [ABSTRACT FROM AUTHOR]

Few-shot learning aims to learn a classifier with more generalization capability from extremely limited labeled samples has drawn an increasing amount of attention in many areas. One typical work in this field is the transductive propagation network (TPN), which propagates labels by capturing the local geometry distribution information between data. However, TPN exploited only the manifold structure of feature extractor networks' high-layer semantic features (single-scale) distribution, while neglecting their local geometry distribution in the low-layer semantic features. This paper presents the adaptive multi-scale transductive information propagation (AMTIP) model to address this problem. Specifically, we introduce the multi-scale feature extractor networks to simultaneously learn samples' high-layer global semantic features and low-layer local semantic features. After acquiring the multi-scale features, we propose the adaptive multi-scale fusion networks to generate the adaptive semantic fusion features applying to different few-shot classification tasks. Finally, the adaptive semantic fusion features are applied to the label propagation model for few-shot image classification. Compared with the single-scale semantic features of TPN, AMTIP can better preserve the local geometry between the support set and query set data via the adaptive multi-scale semantic fusion features. Extensive experiments demonstrate the superiority of the proposed AMTIP compared to the state-of-the-art few-shot classification models. [ABSTRACT FROM AUTHOR]

• To our best knowledge, the common ground of mutual promotion and conflict in joint semantic and instance segmentation are firstly analyzed in detail in this work. • The similarity-based feature fusion module could maintain discriminative features and characterize inconspicuous content. • The cross-task probability-based feature fusion module could model the task-relatedness by establishing probabilistic correlation between semantic and instance features. • The proposed method significantly outperforms state-of-the-arts in both semantic and instance segmentation on two benchmarks. In this paper, we propose a novel method named JSPNet, to segment 3D point cloud in semantic and instance simultaneously. First, we analyze the problem in addressing joint semantic and instance segmentation, including the common ground of cooperation of two tasks, conflict of two tasks, quadruplet relation between semantic and instance distributions, and ignorance of existing works. Then we introduce our method to reinforce mutual cooperation and alleviate the essential conflict. Our method has a shared encoder and two decoders to address two tasks. Specifically, to maintain discriminative features and characterize inconspicuous content, a similarity-based feature fusion module is designed to locate the inconspicuous area in the feature of current branch and then select related features from the other branch to compensate for the unclear content. Furthermore, given the salient semantic feature and the salient instance feature, a cross-task probability-based feature fusion module is developed to establish the probabilistic correlation between semantic and instance features. This module could transform features from one branch and further fuse them with the other branch by multiplying probabilistic matrix. Experimental results on a large-scale 3D indoor point cloud dataset S3DIS and a part-segmentation dataset ShapeNet have demonstrated the superiority of our method over existing state-of-the-arts in both semantic and instance segmentation. The proposed method outperforms PointNet with 12% and 26% improvements and outperforms ASIS with 2.7% and 4.3% improvements in terms of mIoU and mPre. Code of this work has been made available at https://github.com/Chenfeng1271/JSPNet. [ABSTRACT FROM AUTHOR]

• The illumination recovery model converts severe varying illumination to slight/moderate varying illumination for the deep learning feature. • The gradient descent algorithm is employed to tackle the illumination recovery model. • The GRI is generated by normalizing singular values of the logarithm version of the severe illumination variation face image to have unit L2-norm. • The GRIR preserves better face inherent information than the GRI. The deep learning feature is the best for face recognition nowadays, but its performance exhibits unsatisfactorily under severe illumination variations. The main reason is that the deep learning feature was trained by the internet face images with variations of large pose/expression and slight/moderate illumination, which cannot well tackle severe illumination variations. Inspired by the fact that the deep learning feature can cope well with slight/moderate varying illumination, this paper proposes an illumination recovery model to transform severe varying illumination to slight/moderate varying illumination. The illumination recovery model enables the illumination of the severe illumination variation image close to that of the reference image with slight/moderate varying illumination. The reference image generated from the severe illumination variation image is termed as the generated reference image (GRI), which is obtained by normalizing singular values of the logarithm version of the severe illumination variation image to have unit L2-norm. The gradient descent algorithm is employed to address the proposed illumination recovery model, to obtain the generated reference image based illumination recovery image (GRIR). GRIR preserves better face inherent information than GRI such as the face color. Experimental results indicate that the proposed GRIR can efficiently improve the performance of the deep learning feature under severe illumination variations. [ABSTRACT FROM AUTHOR]

Unsupervised graph-structure learning (GSL) which aims to learn an effective graph structure applied to arbitrary downstream tasks by data itself without any labels' guidance, has recently received increasing attention in various real applications. Although several existing unsupervised GSL has achieved superior performance in different graph analytical tasks, how to utilize the popular graph masked autoencoder to sufficiently acquire effective supervision information from the data itself for improving the effectiveness of learned graph structure has been not effectively explored so far. To tackle the above issue, we present a multilevel contrastive graph masked autoencoder (MCGMAE) for unsupervised GSL. Specifically, we first introduce a graph masked autoencoder with the dual feature masking strategy to reconstruct the same input graph-structured data under the original structure generated by the data itself and learned graph-structure scenarios, respectively. And then, the inter-and intra-class contrastive loss is introduced to maximize the mutual information in feature and graph-structure reconstruction levels simultaneously. More importantly, the above inter-and intra-class contrastive loss is also applied to the graph encoder module for further strengthening their agreement at the feature-encoder level. In comparison to the existing unsupervised GSL, our proposed MCGMAE can effectively improve the training robustness of the unsupervised GSL via different-level supervision information from the data itself. Extensive experiments on three graph analytical tasks and eight datasets validate the effectiveness of the proposed MCGMAE.

Background: As an irreversible post-translational modification, protein carbonylation is closely related to many diseases and aging. Protein carbonylation prediction for related patients is significant, which can help clinicians make appropriate therapeutic schemes. Because carbonylation sites can be used to indicate change or loss of protein function, integrating these protein carbonylation site data has been a promising method in prediction. Based on these protein carbonylation site data, some protein carbonylation prediction methods have been proposed. However, most data is highly class imbalanced, and the number of un-carbonylation sites greatly exceeds that of carbonylation sites. Unfortunately, existing methods have not addressed this issue adequately., Results: In this work, we propose a novel two-way rebalancing strategy based on the attention technique and generative adversarial network (Carsite_AGan) for identifying protein carbonylation sites. Specifically, Carsite_AGan proposes a novel undersampling method based on attention technology that allows sites with high importance value to be selected from un-carbonylation sites. The attention technique can obtain the value of each sample's importance. In the meanwhile, Carsite_AGan designs a generative adversarial network-based oversampling method to generate high-feasibility carbonylation sites. The generative adversarial network can generate high-feasibility samples through its generator and discriminator. Finally, we use a classifier like a nonlinear support vector machine to identify protein carbonylation sites., Conclusions: Experimental results demonstrate that our approach significantly outperforms other resampling methods. Using our approach to resampling carbonylation data can significantly improve the effect of identifying protein carbonylation sites., (© 2023. The Author(s).)

Background: Cancer is one of the leading death causes around the world. Accurate prediction of its survival time is significant, which can help clinicians make appropriate therapeutic schemes. Cancer data can be characterized by varied molecular features, clinical behaviors and morphological appearances. However, the cancer heterogeneity problem usually makes patient samples with different risks (i.e., short and long survival time) inseparable, thereby causing unsatisfactory prediction results. Clinical studies have shown that genetic data tends to contain more molecular biomarkers associated with cancer, and hence integrating multi-type genetic data may be a feasible way to deal with cancer heterogeneity. Although multi-type gene data have been used in the existing work, how to learn more effective features for cancer survival prediction has not been well studied., Results: To this end, we propose a deep learning approach to reduce the negative impact of cancer heterogeneity and improve the cancer survival prediction effect. It represents each type of genetic data as the shared and specific features, which can capture the consensus and complementary information among all types of data. We collect mRNA expression, DNA methylation and microRNA expression data for four cancers to conduct experiments., Conclusions: Experimental results demonstrate that our approach substantially outperforms established integrative methods and is effective for cancer survival prediction., Availability and Implementation: https://github.com/githyr/ComprehensiveSurvival ., (© 2023. The Author(s).)

In real-world applications, we often encounter multi-view learning tasks where we need to learn from multiple sources of data or use multiple sources of data to make decisions. Multi-view representation learning, which can learn a unified representation from multiple data sources, is a key pre-task of multi-view learning and plays a significant role in real-world applications. Accordingly, how to improve the performance of multi-view representation learning is an important issue. In this work, inspired by human collective intelligence shown in group decision making, we introduce the concept of view communication into multi-view representation learning. Furthermore, by simulating human communication mechanism, we propose a novel multi-view representation learning approach that can fulfill multi-round view communication. Thus, each view of our approach can exploit the complementary information from other views to help with modeling its own representation, and mutual help between views is achieved. Extensive experiment results on six datasets from three significant fields indicate that our approach substantially improves the average classification accuracy by 4.536% in medicine and bioinformatics fields as well as 4.115% in machine learning field.

Background: As a highly aggressive disease, cancer has been becoming the leading death cause around the world. Accurate prediction of the survival expectancy for cancer patients is significant, which can help clinicians make appropriate therapeutic schemes. With the high-throughput sequencing technology becoming more and more cost-effective, integrating multi-type genome-wide data has been a promising method in cancer survival prediction. Based on these genomic data, some data-integration methods for cancer survival prediction have been proposed. However, existing methods fail to simultaneously utilize feature information and structure information of multi-type genome-wide data., Results: We propose a Multi-type Data Joint Learning (MDJL) approach based on multi-type genome-wide data, which comprehensively exploits feature information and structure information. Specifically, MDJL exploits correlation representations between any two data types by cross-correlation calculation for learning discriminant features. Moreover, based on the learned multiple correlation representations, MDJL constructs sample similarity matrices for capturing global and local structures across different data types. With the learned discriminant representation matrix and fused similarity matrix, MDJL constructs graph convolutional network with Cox loss for survival prediction., Conclusions: Experimental results demonstrate that our approach substantially outperforms established integrative methods and is effective for cancer survival prediction., (© 2022. The Author(s).)

In this paper, we present a novel high-frequency facial feature and a high-frequency based sparse representation classification to tackle single sample face recognition (SSFR) under varying illumination. Firstly, we propose the assumption that QRCP bases can represent intrinsic face surface features with different frequencies, and their corresponding energy coefficients describe illumination intensities. Based on this assumption, we take QRCP bases with corresponding weighting coefficients (i.e. the major components of energy coefficients) to develop the high-frequency facial feature of the face image, which is named as QRCP-face. The normalized QRCP-face (NQRCPface) is constructed to further constraint illumination effects by normalizing the weighting coefficients of QRCP-face. Moreover, we propose the adaptive QRCP-face (AQRCP-face) that assigns a special parameter to NQRCP-face via the illumination level estimated by the weighting coefficients. Secondly, we consider that the differences of pixel images cannot model the intraclass variations of generic faces with illumination variations, and the specific identification information of the generic face is redundant for the current SSFR with generic learning. To tackle above two issues, we develop a general high-frequency based sparse representation (GHSP) model. Two practical approaches separated high-frequency based sparse representation (SHSP) and unified high-frequency based sparse representation (UHSP) are developed. Finally, the performances of the proposed methods are verified on the Extended Yale B, CMU PIE, AR, LFW and our self-built Driver face databases. The experimental results indicate that the proposed methods outperform previous approaches for SSFR under varying illumination.

Linear discrimination analysis (LDA) technique is an important and well-developed area of image recognition and to date many linear discrimination methods have been put forward. Despite these efforts, there persist in LDA at least three areas of weakness. The first weakness is that not all the discrimination vectors that are obtained are useful in pattern classification. Second, it remains computationally expensive to make the discrimination vectors completely satisfy statistical uncorrelation. The third weakness is that it is necessary to select the appropriate principal components. In this paper, we propose to improve discrimination technique in these three areas and to that end present an improved LDA (ILDA) approach which synthesizes these improvements. Experimental results on different image databases demonstrate that our improvements on LDA are efficient, and that ILDA outperforms other state-of-the-art linear discrimination methods.