Search

In most traditional machine learning algorithms, the training and testing datasets have identical distributions and feature spaces. However, these assumptions have not held in many real applications. Although transfer learning methods have been invented to fill this gap, they introduce new challenges as negative transfers (NTs). Most previous research considered NT a significant problem, but they pay less attention to solving it. This study will propose a transductive learning algorithm based on cellular learning automata (CLA) to alleviate the NT issue. Two famous learning automata (LA) entitled estimators are applied as estimator CLA in the proposed algorithms. A couple of new decision criteria called merit and and attitude parameters are introduced to CLA to limit NT. The proposed algorithms are applied to standard LA environments. The experiments show that the proposed algorithm leads to higher accuracy and less NT results.

Some researchers have introduced transfer learning mechanisms to multiagent reinforcement learning (MARL). However, the existing works devoted to cross-task transfer for multiagent systems were designed just for homogeneous agents or similar domains. This work proposes an all-purpose cross-transfer method, called multiagent lateral transfer (MALT), assisting MARL with alleviating the training burden. We discuss several challenges in developing an all-purpose multiagent cross-task transfer learning method and provide a feasible way of reusing knowledge for MARL. In the developed method, we take features as the transfer object rather than policies or experiences, inspired by the progressive network. To achieve more efficient transfer, we assign pretrained policy networks for agents based on clustering, while an attention module is introduced to enhance the transfer framework. The proposed method has no strict requirements for the source task and target task. Compared with the existing works, our method can transfer knowledge among heterogeneous agents and also avoid negative transfer in the case of fully different tasks. As far as we know, this article is the first work denoted to all-purpose cross-task transfer for MARL. Several experiments in various scenarios have been conducted to compare the performance of the proposed method with baselines. The results demonstrate that the method is sufficiently flexible for most settings, including cooperative, competitive, homogeneous, and heterogeneous configurations.

Reinforcement learning (RL) plays an essential role in the field of artificial intelligence but suffers from data inefficiency and model-shift issues. One possible solution to deal with such issues is to exploit transfer learning. However, interpretability problems and negative transfer may occur without explainable models. In this article, we define Relation Transfer as explainable and transferable learning based on graphical model representations, inferring the skeleton and relations among variables in a causal view and generalizing to the target domain. The proposed algorithm consists of the following three steps. First, we leverage a suitable casual discovery method to identify the causal graph based on the augmented source domain data. After that, we make inferences on the target model based on the prior causal knowledge. Finally, offline RL training on the target model is utilized as prior knowledge to improve the policy training in the target domain. The proposed method can answer the question of what to transfer and realize zero-shot transfer across related domains in a principled way. To demonstrate the robustness of the proposed framework, we conduct experiments on four classical control problems as well as one simulation to the real-world application. Experimental results on both continuous and discrete cases demonstrate the efficacy of the proposed method.

Existing transfer learning methods that focus on problems in stationary environments are not usually applicable to dynamic environments, where concept drift may occur. To the best of our knowledge, the concept drift-tolerant transfer learning (CDTL), whose major challenge is the need to adapt the target model and knowledge of source domains to the changing environments, has yet to be well explored in the literature. This article, therefore, proposes a hybrid ensemble approach to deal with the CDTL problem provided that data in the target domain are generated in a streaming chunk-by-chunk manner from nonstationary environments. At each time step, a class-wise weighted ensemble is presented to adapt the model of target domains to new environments. It assigns a weight vector for each classifier generated from the previous data chunks to allow each class of the current data leveraging historical knowledge independently. Then, a domain-wise weighted ensemble is introduced to combine the source and target models to select useful knowledge of each domain. The source models are updated with the source instances performed by the proposed adaptive weighted CORrelation ALignment (AW-CORAL). AW-CORAL iteratively minimizes domain discrepancy meanwhile decreases the effect of unrelated source instances. In this way, positive knowledge of source domains can be potentially promoted while negative knowledge is reduced. Empirical studies on synthetic and real benchmark data sets demonstrate the effectiveness of the proposed algorithm.

The research of semantic segmentation based on unsupervised domain adaptation greatly alleviates the high-cost bottleneck of manual annotation in deep learning. Inevitably domain gap limits the ability of target domain to learn knowledge from source domain. Previous domain alignment strategies aim to explore maximum domain-invariant space to enlarge knowledge that target domain learns. They based on the assumption that images are symmetrical in categories covering all predefined categories. However, in practical training, there is class asymmetry in image samples inter domains. This will lead to outlier categories and negative transfer, because it lacks the guidance of the corresponding category of the source domain. To tackle this issue, we propose a partial domain adaptive method on semantic segmentation to guide target model to learn category-level knowledge selectively. PAM (Partial Adaptive Map) module is introduced to motivate the target model to obtain more knowledge from non-outliers and less knowledge from outliers to avoid negative transfer. We further analyze the effectiveness of this method from the perspective of JS divergence. Our method has achieved significant improvement without additional discriminators. Experiments on general datasets GTA5 and SYNTHIA can compare with SOTA methods.

Transfer learning algorithm can provide a framework to utilize the previous knowledge to train fuzzy neural network (FNN). However, the performance of TL-based FNN will be destroyed by the knowledge over-fitting problem in the learning process. To solve this problem, an interactive transfer learning (ITL) algorithm, which can alleviate the negative transfer among different domains to improve the learning performance of FNN, is designed and analyzed in this paper. This ITL-assisted FNN (ITL-FNN) contains the following advantages. Frist, a knowledge filter algorithm is developed to reconstruct the knowledge in source scene by balancing the matching accuracy and diversity. Then, the knowledge from source scene can fit the instance of target scene with suitable accuracy. Second, a self-balancing mechanism is designed to balance the driven information between the source and target scenes. Then, the knowledge can be refitted to reduce the useless information. Third, a structural competition algorithm is proposed to adjust the knowledge of FNN. Then, the proposed ITL-FNN can achieve compact structure to improve the generalization performance. Finally, some benchmark problems and industrial applications are provided to demonstrate the merits of ITL-FNN.

Evolutionary multi-task optimization (EMTO) has recently attracted widespread attention in the evolutionary computation community, which solves two or more tasks simultaneously to improve the convergence characteristics of tasks when individually optimized. Effective knowledge between tasks is transferred by taking advantage of the parallelism of population-based search. Without any prior knowledge about tasks, it is a challenging problem of how to adaptively transfer effective knowledge between tasks and reduce the impact of negative transfer in EMTO. However, these two issues are rarely studied simultaneously in the existing literature. Besides, in complex many-task environments, the potential relationships among individuals from highly diverse populations associated with tasks directly determine the effectiveness of cross-task knowledge transfer. Keeping those in mind, we propose a multi-task evolutionary algorithm based on anomaly detection (MTEA-AD). Specifically, each task is assigned a population and an anomaly detection model. Each anomaly detection model is used to learn the relationship among individuals between the current task and the other tasks online. Individuals that may carry negative knowledge are identified as outliers. And candidate transferred individuals identified by the anomaly detection model are selected to assist the current task, which may carry common knowledge across the current task and other tasks. Furthermore, to realize the adaptive control of the degree of knowledge transfer, the successfully transferred individuals that survive to the next generation through the elitism are used to update the anomaly detection parameter. The fair competition between offspring and candidate transferred individuals can effectively reduce the risk of negative transfer. Finally, the empirical studies on a series of synthetic benchmarks and a practical study are conducted to verify the effectiveness of MTEA-AD. The experimental results demonstrate that our proposal can adaptively adjust the degree of knowledge transfer through the anomaly detection model to achieve highly competitive performance compared to several state-of-the-art EMTO methods.

Multi-task optimization aims to solve two or more optimization tasks simultaneously by leveraging inter-task knowledge transfer. However, as the number of tasks increases to the extent of many-task optimization, the knowledge transfer between tasks encounters more uncertainty and challenges, thereby resulting in degradation of optimization performance. To give full play to the many-task optimization framework and minimize the potential negative transfer, this paper proposes an evolutionary many-task optimization algorithm based on a multi-source knowledge transfer mechanism namely EMaTO-MKT. Particularly, in each iteration, EMaTO-MKT determines the probability of using knowledge transfer adaptively according to the evolution experience, and balances the self-evolution within each task and the knowledge transfer among tasks. To perform knowledge transfer, EMaTO-MKT selects multiple highly similar tasks in term of maximum mean discrepancy as the learning sources for each task. Afterward, a knowledge transfer strategy based on local distribution estimation is applied to enable the learning from multiple sources. Compared with the other state-of-the-art evolutionary many-task algorithms on benchmark test suites, EMaTO-MKT shows competitiveness in solving many-task optimization problems.

As an effective learning paradigm against insufficient training samples, Multi-Task Learning (MTL) encourages knowledge sharing across multiple related tasks so as to improve the overall performance. In MTL, a major challenge springs from the phenomenon that sharing the knowledge with dissimilar and hard tasks, known as negative transfer, often results in a worsened performance. Though a substantial amount of studies have been carried out against the negative transfer, most of the existing methods only model the transfer relationship as task correlations, with the transfer across features and tasks left unconsidered. Different from the existing methods, our goal is to alleviate negative transfer collaboratively across features and tasks. To this end, we propose a novel multi-task learning method called Task-Feature Collaborative Learning (TFCL). Specifically, we first propose a base model with a heterogeneous block-diagonal structure regularizer to leverage the collaborative grouping of features and tasks and suppressing inter-group knowledge sharing. We then propose an optimization method for the model. Extensive theoretical analysis shows that our proposed method has the following benefits: (a) it enjoys the global convergence property and (b) it provides a block-diagonal structure recovery guarantee. As a practical extension, we extend the base model by allowing overlapping features and differentiating the hard tasks. We further apply it to the personalized attribute prediction problem with fine-grained modeling of user behaviors. Finally, experimental results on both simulated dataset and real-world datasets demonstrate the effectiveness of our proposed method, Comment: To appear in T-PAMI

Among different recommendation techniques, collaborative filtering usually suffers from limited performance due to the data sparsity problem. Transfer learning presents an unprecedented opportunity to alleviate this issue by transferring rating patterns from the source domain to the target domain. However, an issue occurs in some situations when the source and target domains have unrelated parts to each other, such that a negative transfer may reduce the learning performance in the target domain. To address this issue, in this article, we present a novel model called cluster rating bias restoration to divide the learning process into two stages. First, we employ traditional transfer learning methods to adapt and reconstruct the target rating matrix. Second, we propose a cluster rating bias restoration framework specifically to minimize the effect of the negative transfer from the source domain and to restore the context-information-based rating bias trend in the target domain. Experiments on real-world datasets show that our proposed model outperforms the state-of-the-art alternatives.

Motivation Large-scale cancer omics studies have highlighted the diversity of patient molecular profiles and the importance of leveraging this information to deliver the right drug to the right patient at the right time. Key challenges in learning predictive models for this include the high-dimensionality of omics data and heterogeneity in biological and clinical factors affecting patient response. The use of multi-task learning techniques has been widely explored to address dataset limitations for in vitro drug response models, while domain adaptation (DA) has been employed to extend them to predict in vivo response. In both of these transfer learning settings, noisy data for some tasks (or domains) can substantially reduce the performance for others compared to single-task (domain) learners, i.e. lead to negative transfer (NT). Results We describe a novel multi-task unsupervised DA method (TUGDA) that addresses these limitations in a unified framework by quantifying uncertainty in predictors and weighting their influence on shared feature representations. TUGDA’s ability to rely more on predictors with low-uncertainty allowed it to notably reduce cases of NT for in vitro models (94% overall) compared to state-of-the-art methods. For DA to in vivo settings, TUGDA improved over previous methods for patient-derived xenografts (9 out of 14 drugs) as well as patient datasets (significant associations in 9 out of 22 drugs). TUGDA’s ability to avoid NT thus provides a key capability as we try to integrate diverse drug-response datasets to build consistent predictive models with in vivo utility. Availabilityand implementation https://github.com/CSB5/TUGDA. Supplementary information Supplementary data are available at Bioinformatics online.

Despite recent remarkable progress, person re-identification (ReID) still suffers from a shortage of annotated training data. To deal with this problem, there has been a boost of interest in developing various data augmentation methods. In this paper, we are devoted to developing an end-to-end joint representation learning framework for the ReID task on the basis of a novel data augmentation strategy. Specifically, we regard the original training dataset as a source domain and generate the counterpart augmented domains through image channel shuffling. Accordingly, we design a symmetric classification network for ReID learning. By investigating the domain-level and identity-level relationship between domains, we use the idea of negative transfer and structural consistency to optimize the network for learning discriminative feature embeddings. Comprehensive experiments on some benchmark datasets demonstrate the effectiveness and robustness of our proposed approach. Source code is released at: https://github.com/flychen321/negative_transfer_reid .

Gaussian mixture model (GMM) is a well-known model-based approach for data clustering. However, when the data samples are insufficient, the classical GMM-based clustering algorithms are not effective anymore. Referring to the idea of transfer clustering methods, this paper proposes a general transfer GMM-based clustering framework, which employs the important knowledge extracted from some known source domain to guide and improve the clustering on the target domain with small-scale data. Specifically, three traditional GMM-based clustering approaches are extended to the corresponding transfer clustering versions. Furthermore, to avoid the negative transfer problem, maximum mean discrepancy (MMD) is introduced to search the most matched source domain to provide more positive guidance for data clustering on the target domain. Experiments on synthetic and real-world datasets demonstrate the efficiency of the presented framework compared with several existing transfer clustering algorithms.

Convolutional neural networks have demonstrated state-of-the-art performance in image classification and various other computer vision tasks. Plant disease detection is an important area of deep learning which has been addressed by many recent methods. However, there is a dire need to optimize these solutions for resource-constrained portable devices such as smartphones. This is a challenging problem because deep learning models are resource extensive in nature. This paper proposes an efficient method to systematically classify plant disease symptoms using convolutional neural networks. These networks are memory efficient and when coupled with the proposed training configuration it enables rapid development of industrial applications by reducing the training times. Another critical problem arises with the improper distribution of samples among classes known as the class imbalance problem, which is addressed by employing a simple statistical methodology. Transfer learning is a known technique for training small datasets which transfers pre-trained weights learned on a large dataset. However, during transfer learning, negative transfer learning is a common problem. Therefore, a stepwise transfer learning approach is proposed which can help in fast convergence, while reducing overfitting and preventing negative transfer learning during knowledge transfer across domains. The system is trained and evaluated on two plant disease datasets i.e., PlantVillage (a publicly available dataset) and pepper disease dataset provided by the National Institute of Horticultural and Herbal Science, Republic of Korea. The pepper dataset is particularly challenging as it contains images from different parts of the plant such as the leaf, pulp, and stem. The proposed system has dominated the previous works on the PlantVillage dataset and achieved 99% and 99.69% accuracy on the Pepper dataset and PlantVillage datasets, respectively.

Open Set Domain Adaptation (OSDA) focuses on bridging the domain gap between a labeled source domain and an unlabeled target domain, while also rejecting target classes that are not present in the source as unknown. The challenges of this task are closely related to those of Positive-Unlabeled (PU) learning where it is essential to discriminate between positive (known) and negative (unknown) class samples in the unlabeled target data. With this newly discovered connection, we leverage the theoretical framework of PU learning for OSDA and, at the same time, we extend PU learning to tackle uneven data distributions. Our method combines domain adversarial learning with a new non-negative risk estimator for PU learning based on self-supervised sample reconstruction. With experiments on digit recognition and object classification, we validate our risk estimator and demonstrate that our approach allows reducing the domain gap without suffering from negative transfer.

In this paper we propose two methods for instance transfer based on conformal prediction. As a distinctive character, both of the methods are model independent and combine feature selection and source-instance selection to avoid negative transfer. The methods have been tested experimentally for different types of classification model on several benchmark data sets. The experimental results demonstrate that the new methods are capable of outperforming significantly standard instance transfer methods. (C) 2019 Elsevier B.V. All rights reserved.

The transfer learning methodology leverages knowledge from the source domain with abundant training data to the insufficient target domain. Recently, new approaches continue to be developed and used to solve different classification tasks, ranging from public news to videos and to many others. Most transfer learning methods are based on the assumption that both source and target data are in the same feature space or with the same data distribution, which however is not always true in real applications where it would lead to a negative transfer. In order to overcome this hurdle, the multiple-source transfer learning framework is useful. Since many real systems can be represented by networks, how to utilize the structural similarity between different networks so as to increase the transfer effectiveness becomes important. In this paper, the NES specification index is used to quantitatively measure the structural similarity between two networks, based on which a new transfer learning method (named NES-TL) is developed. Experiments on tag popularity prediction in StackExchange Q&A communities verify the effectiveness of the proposed approach, showing that it behaves better than existing baseline methods.

Person re-identification (Re-ID) across multiple datasets is a challenging task due to two main reasons: the presence of large cross-dataset distinctions and the absence of annotated target instances. To address these two issues, this paper proposes a domain adaptive attention learning approach to reliably transfer discriminative representation from the labeled source domain to the unlabeled target domain. In this approach, a domain adaptive attention model is learned to separate the feature map into domain-shared part and domain-specific part. In this manner, the domain-shared part is used to capture transferable cues that can compensate cross-dataset distinctions and give positive contributions to the target task, while the domain-specific part aims to model the noisy information to avoid the negative transfer caused by domain diversity. A soft label loss is further employed to take full use of unlabeled target data by estimating pseudo labels. Extensive experiments on the Market-1501, DukeMTMC-reID and MSMT17 benchmarks demonstrate the proposed approach outperforms the state-of-the-arts.

Humans rarely tackle every problem from scratch. Given this observation, the motivation for this paper is to improve optimization performance through adaptive knowledge transfer across related problems. The scope for spontaneous transfers under the simultaneous occurrence of multiple problems unveils the benefits of multitasking. Multitask optimization has recently demonstrated competence in solving multiple (related) optimization tasks concurrently. Notably, in the presence of underlying relationships between problems, the transfer of high-quality solutions across them has shown to facilitate superior performance characteristics. However, in the absence of any prior knowledge about the intertask synergies (as is often the case with general black-box optimization), the threat of predominantly negative transfer prevails. Susceptibility to negative intertask interactions can impede the overall convergence behavior. To allay such fears, in this paper, we propose a novel evolutionary computation framework that enables online learning and exploitation of the similarities (and discrepancies) between distinct tasks in multitask settings, for an enhanced optimization process. Our proposal is based on the principled theoretical arguments that seek to minimize the tendency of harmful interactions between tasks, based on a purely data-driven learning of relationships among them. The efficacy of our proposed method is validated experimentally on a series of synthetic benchmarks, as well as a practical study that provides insights into the behavior of the method in the face of several tasks occurring at once. Accepted version

The human brain has been a main source of inspiration for designing deep learning models. Recently, inspired by the specialized functions of two cerebral hemispheres in processing low and high spatial frequency information, some dual-path neural networks with global and local branches have been proposed to deal with both coarse- and fine-grained visual tasks simultaneously. However, in existing works, the interhemispheric communication mechanism, which is responded by the corpus callosum, the largest white matter structure in the human brain that connecting the left and right cerebral hemispheres, is still not fully explored and exploited. This paper aims to explore how the corpus callosum can inspire us to enable transfer and integration of information between global and local branches in hemisphere-inspired artificial neural networks, such that one branch can leverage the other's learned knowledge and benefit each other. To this end, we propose a gated intercommunication unit to selectively transfer useful knowledge between the two branches via attention mechanisms to alleviate the negative transfer. Experiments on sb-MNIST and two pedestrian attribute datasets show that the proposed method outperforms the compared ones in most cases.

Unsupervised domain adaptation without accessing expensive annotation processes of target data has achieved remarkable successes in semantic segmentation. However, most existing state-of-the-art methods cannot explore whether semantic representations across domains are transferable or not, which may result in the negative transfer brought by irrelevant knowledge. To tackle this challenge, in this paper, we develop a novel Knowledge Aggregation-induced Transferability Perception (KATP) for unsupervised domain adaptation, which is a pioneering attempt to distinguish transferable or untransferable knowledge across domains. Specifically, the KATP module is designed to quantify which semantic knowledge across domains is transferable, by incorporating transferability information propagation from global category-wise prototypes. Based on KATP, we design a novel KATP Adaptation Network (KATPAN) to determine where and how to transfer. The KATPAN contains a transferable appearance translation module T_A() and a transferable representation augmentation module T_R(), where both modules construct a virtuous circle of performance promotion. T_A() develops a transferability-aware information bottleneck to highlight where to adapt transferable visual characterizations and modality information; T_R() explores how to augment transferable representations while abandoning untransferable information, and promotes the translation performance of T_A() in return. Experiments on several representative datasets and a medical dataset support the state-of-the-art performance of our model.

Incorporating brain-computer interfaces (BCIs) into daily life requires reducing the reliance of decoding algorithms on the calibration or enabling calibration with the minimal burden on the user. A potential solution could be a pre-trained decoder demonstrating a reasonable accuracy on the naive operators. Addressing this issue, we considered ambiguous stimuli classification tasks and trained an artificial neural network to classify brain responses to the stimuli of low and high ambiguity. We built a pre-trained classifier utilizing time-frequency features corresponding to the fundamental neurophysiological processes shared between subjects. To extract these features, we statistically contrasted electroencephalographic (EEG) spectral power between the classes in the representative group of subjects. As a result, the pre-trained classifier achieved 74% accuracy on the data of newly recruited subjects. Analysis of the literature suggested that a pre-trained classifier could help naive users to start using BCI bypassing training and further increased accuracy during the feedback session. Thus, our results contribute to using BCI during paralysis or limb amputation when there is no explicit user-generated kinematic output to properly train a decoder. In machine learning, our approach may facilitate the development of transfer learning (TL) methods for addressing the cross-subject problem. It allows extracting the interpretable feature subspace from the source data (the representative group of subjects) related to the target data (a naive user), preventing the negative transfer in the cross-subject tasks.

Amdo Tibetan is a silent tone language. For college students who take Amdo Tibetan as their mother tongue, the tone of Mandarin has always been a major difficulty in their Mandarin learning. In the field of artificial intelligence and Mandarin speech recognition, the speech recognition of Tibetan native speakers is the focus of the current research. Based on this, this paper takes the college students whose native language is Amdo Tibetan as the experimental subjects, extracts the fundamental frequency signals of their Mandarin pronunciation through the experiment of acoustics of speech, and converts the fundamental frequency into the value of 5° through normalization processing. The results show that the main error of their pronunciations is in the tone value, and there is little difference of the tone pattern between their pronunciations and the standard Mandarin. The main cause of their pronunciation errors is mainly affected by the negative transfer of their mother tongue. The research results can be used for reference in the fields of Mandarin teaching, speech recognition and artificial intelligence.

Standard English pronunciation and the ability of English speech recognition are of great significance to English learning. If the foundation of English pronunciation is weak, it will directly have a negative impact on daily English communication and English listening. Therefore, phonetic teaching occupies an important position in English teaching. But many factors affect the pronunciation of English learners. One of the factors that cannot be ignored is the negative transfer of dialects to English learning. With the help of artificial intelligence technology, this study puts forward some suggestions on how to overcome the negative transfer of dialects to English phonetic learning in English phonetic teaching.

Laryngeal cancer tumor (LCT) grading is a challenging task in P63 Immunohistochemical (IHC) histopathology images due to small differences between LCT levels in pathology images, the lack of precision in lesion regions of interest (LROIs) and the paucity of LCT pathology image samples. The key to solving the LCT grading problem is to transfer knowledge from other images and to identify more accurate LROIs, but the following problems occur: 1) transferring knowledge without a priori experience often causes negative transfer and creates a heavy workload due to the abundance of image types, and 2) convolutional neural networks (CNNs) constructing deep models by stacking cannot sufficiently identify LROIs, often deviate significantly from the LROIs focused on by experienced pathologists, and are prone to providing misleading second opinions. So we propose a novel fusion attention block network (FABNet) to address these problems. First, we propose a model transfer method based on clinical a priori experience and sample analysis (CPESA) that analyzes the transfer ability by integrating clinical a priori experience using indicators such as the relationship between the cancer onset location and morphology and the texture and staining degree of cell nuclei in histopathology images; our method further validates these indicators by the probability distribution of cancer image samples. Then, we propose a fusion attention block (FAB) structure, which can both provide an advanced non-uniform sparse representation of images and extract spatial relationship information between nuclei; consequently, the LROI can be more accurate and more relevant to pathologists. We conducted extensive experiments, compared with the best Baseline model, the classification accuracy is improved 25%, and It is demonstrated that FABNet performs better on different cancer pathology image datasets and outperforms other state of the art (SOTA) models.

Pretraining and multitask learning are widely used to improve the speech to text translation performance. In this study, we are interested in training a speech to text translation model along with an auxiliary text to text translation task. We conduct a detailed analysis to understand the impact of the auxiliary task on the primary task within the multitask learning framework. Our analysis confirms that multitask learning tends to generate similar decoder representations from different modalities and preserve more information from the pretrained text translation modules. We observe minimal negative transfer effect between the two tasks and sharing more parameters is helpful to transfer knowledge from the text task to the speech task. The analysis also reveals that the modality representation difference at the top decoder layers is still not negligible, and those layers are critical for the translation quality. Inspired by these findings, we propose three methods to improve translation quality. First, a parameter sharing and initialization strategy is proposed to enhance information sharing between the tasks. Second, a novel attention-based regularization is proposed for the encoders and pulls the representations from different modalities closer. Third, an online knowledge distillation is proposed to enhance the knowledge transfer from the text to the speech task. Our experiments show that the proposed approach improves translation performance by more than 2 BLEU over a strong baseline and achieves state-of-the-art results on the \textsc{MuST-C} English-German, English-French and English-Spanish language pairs., Accepted by ACL 2021

Multi-task learning(MTL) is an open and challenging problem in various real-world applications. The typical way of conducting multi-task learning is establishing some global parameter sharing mechanism across all tasks or assigning each task an individual set of parameters with cross-connections between tasks. However, for most existing approaches, all tasks just thoroughly or proportionally share all the features without distinguishing the helpfulness of them. By that, some tasks would be intervened by the unhelpful features that are useful for other tasks, leading to undesired negative transfer between tasks. In this paper, we design a novel architecture named the Multiple-level Sparse Sharing Model (MSSM), which can learn features selectively and share knowledge across all tasks efficiently. MSSM first employs a field-level sparse connection module (FSCM) to enable much more expressive combinations of feature fields to be learned for generalization across tasks while still allowing for task-specific features to be customized for each task. Furthermore, a cell-level sparse sharing module (CSSM) can recognize the sharing pattern through a set of coding variables that selectively choose which cells to route for a given task. Extensive experimental results on several real-world datasets show that MSSM outperforms SOTA models significantly in terms of AUC and LogLoss metrics.

Unsupervised domain adaptation (UDA) aims to transfer knowledge between different domains. Most of the existing UDA methods try to align the conditional distribution between the source and target domains by utilizing the information of pseudo labels induced from the target domain. To tackle the negative transfer caused by inaccurate pseudo labels, we propose a novel UDA method named progressive distribution alignment based on label correction (PDALC). Specifically, PDALC uses the class discriminative information to perform subspace learning to obtain the domain invariance subspace. Furthermore, a new mechanism of pseudo label correction is introduced to measure the reliability of pseudo labels and utmostly correct the inaccurate pseudo labels. By combining subspace learning with label correction, the performance of PDALC can be continuously improved, which in turn reduces the generation of inaccurate pseudo labels. The experimental results show that the proposed method outperforms the state-of-the-art UDA methods.

Human activity recognition (HAR) aims at recognizing activities by training models on the large quantity of sensor data. Since it is time-consuming and expensive to acquire abundant labeled data, transfer learning becomes necessary for HAR by transferring knowledge from existing domains. However, there are two challenges existing in cross-dataset activity recognition. The first challenge is source domain selection. Given a target task and several available source domains, it is difficult to determine how to select the most similar source domain to the target domain such that negative transfer can be avoided. The second one is accurately activity transfer. After source domain selection, how to achieve accurate knowledge transfer between the selected source and the target domain remains another challenge. In this paper, we propose an Adaptive Spatial-Temporal Transfer Learning (ASTTL) approach to tackle both of the above two challenges in cross-dataset HAR. ASTTL learns the spatial features in transfer learning by adaptively evaluating the relative importance between the marginal and conditional probability distributions. Besides, it captures the temporal features via incremental manifold learning. Therefore, ASTTL can learn the adaptive spatial-temporal features for cross-dataset HAR and can be used for both source domain selection and accurate activity transfer. We evaluate the performance of ASTTL through extensive experiments on 4 public HAR datasets, which demonstrates its effectiveness. Furthermore, based on ASTTL, we design and implement an adaptive cross-dataset HAR system called Client-Cloud Collaborative Adaptive Activity Recognition System (3C2ARS) to perform HAR in the real environment. By collecting activities in the smartphone and transferring knowledge in the cloud server, ASTTL can significantly improve the performance of source domain selection and accurate activity transfer.

Satellite image classification plays an important role in many fields. It can be divided into two groups: supervised and unsupervised classification. The former requires a large number of labeled data, while, in practice, satellite images usually lack of sufficient ones. How to achieve high accuracy on unsupervised satellite image classification is a key problem. For tackling this problem, based on the idea of partial transfer learning, we propose an novel end-to-end unsupervised classification method called coordinate partial adversarial domain adaptation (CPADA) for satellite image classification. Under the aid of a novel coordinate loss, our framework transfers relevant examples in the shared classes to promote performance, and ignore irrelevant ones in the specific classes to mitigate negative transfer. Experiments show that our CPADA exceeds state-of-the-art results for unsupervised satellite image classification task.

Previous studies on ESL/EFL learners’ verb-noun (V-N) miscollocations have shed some light on common miscollocation types and possible causes. However, barriers to further understanding of learners’ difficulties still exist, such as the limited amount of learner data generated from small corpora and the labor-intensive process of manually retrieving collocational errors. To provide researchers with a more efficient retrieval method, this study proposed the use of the Sketch-Diff function in the Sketch Engine (SKE) platform to semi-automatically retrieve collocation errors in large learner corpora. To test the feasibility of this semi-automatic retrieval method, a 7.4-million-word EFL learner corpus was investigated with Sketch-Diff, and 4541 tokens of common miscollocations were identified. Analysis of these miscollocations revealed that most errors were verb-based and often caused by negative transfer from the learners’ L1, undergeneralization (e.g., ignorance of L2 syntactic rules), and approximation (e.g., the misuse of near-synonyms, hyper-/hyponyms, antonyms, and lexemes with similar sound/form). This study demonstrates that using Sketch-Diff to retrieve V-N miscollocations from a large learner corpus is both feasible and efficient. This method can be applied to other languages to further deepen our understanding of L2 learners’ difficulties in collocation acquisition.

Unsupervised domain adaptation (UDA) aims to improve the classification performance on an unlabeled target domain by leveraging information from a fully labeled source domain. Recent approaches explore domain-invariant and class-discriminant representations to tackle this task. These methods, however, ignore the interaction between domain alignment learning and class discrimination learning. As a result, the missing or inadequate tradeoff between domain alignment and class discrimination are prone to the problem of negative transfer. In this paper, we propose Dynamic Weighted Learning (DWL) to avoid the discriminability vanishing problem caused by excessive alignment learning and domain misalignment problem caused by excessive discriminant learning. Technically, DWL dynamically weights the learning losses of alignment and discriminability by introducing the degree of alignment and discriminability. Besides, the problem of sample imbalance across domains is first considered in our work, and we solve the problem by weighing the samples to guarantee information balance across domains. Extensive experiments demonstrate that DWL has an excellent performance in several benchmark datasets., Comment: This paper has been accepted by CVPR2021

In order to explore the negative transfer of Guangzhou dialect on the British English pronunciation, this paper takes English major freshmen in Nanfang College of Sun Yat-sen University as respondents to participate into a phonetic test and analyzes their English pronunciation of vowels, consonants, stress, rhythm, and intonation, based on the data collected by a speech analyzer software, Praat and the evaluation done by foreign teachers (i.e. Ear Recognition Method). In conclusion, it is found that the respondents’ pronunciation of segmental phoneme and suprasegmental phoneme was negatively affected by Guangzhou dialect in different levels. In connection with the major findings, the authors provide four suggestions for English learners with Guangzhou dialect as mother tongue to improve their English pronunciation.

In our daily lives, people encounter different problems in pattern recognition applications, such as fingerprint unlocking, faoe recognition, DNA detection, mother tongue culture transfer. The essenoe of pattern recognition is to match the known sample with the given pattern by comparing the known sample with the given pattern, so as to achieve the purpose of classification. In fuzzy sets, membership degree is often used to describe samples and patterns, and use fuzzy measure to calculate the similarity between samples and patterns, so that the overall characteristics of patterns can be well reflected, and samples are allowed to have a certain degree of interference. Sometimes, it is required to deduce the corresponding conclusions from the existing rule base through fuzzy reasoning algorithm, and sort them to get the classification results. Mother tongue transfer is an inevitable problem in English translation, which has a great influence on foreign language learning and translation. Language transfer usually includes four forms: overuse, avoidance, convenience and misuse, which can be divided into positive transfer and negative transfer. Due to the obvious differences between Chinese culture and English culture, negative transfer of mother tongue cannot be avoided, which increases the difficulty of English translation. This paper studies the pattern recognition based culture transfer of machine native language in the network environment.

In the field of medical data mining, collecting sufficient expert labeled datasets especially with pixel-level annotations is extremely difficult. For diseases with scarce data, supervised learning has little effect even with synthetic images for data augmentation or cross-modality knowledge transfer. In response to the problem, we propose a novel solution - knowledge transfer across diseases. Existing transfer methods either focus on mapping one domain to another or focus on generating a subspace consisting of domain invariant features. However, these methods are negative in the transfer of knowledge across diseases because they lose the private characteristics of the target disease. Motivated by this limitation, we present a reweighting network (RW-net). It can be widely applied to a domain adaptation model based on adversarial, through reweighting dynamic balance subspace generation and segmenter optimization. We compare our method to two popular transfer learning methods and baseline on the open glioma dataset (BraTS) and ischemic stroke dataset (ATLAS). Our approach achieves excellent performance, even though the two diseases have completely different imaging traits.

As financial enterprises have moved their services to the internet, financial fraud detection has become an ever-growing problem causing severe economic losses for the financial industry. Recently, machine learning has gained significant attention to handle the financial fraud detection problem as a binary classification problem. While significant progress has been made, fraud detection is still a notable challenge due to two major reasons. First, fraudsters today are adaptive, inventive, and intelligent, making their fraud characteristics are too deep stealth to be detected by simple detection models. Second, labeled samples for training the detection models are usually very few as collecting large-scale training data needs a certain performance-time and is costly. To address the two problems, we propose a novel multi-source transfer learning approach with self-supervised domain distance learning for financial fraud detection problems. The core idea is to transfer relevant knowledge from multiple data-rich sources to the data-poor target task, e.g., learning fraud patterns from several other related mature loan products to improve the fraud detection in a cold-start loan product. Specifically, since the feature distribution discrepancy across domains may cause useless or even negative knowledge transfer, we propose self-supervised domain distance learning under the Wasserstein metric to measure the domain relevance/relationships between target and source tasks. The learned Wasserstein distance helps in selectively transferring most relevant knowledge from source domains to target domains. Thus it reduces the risk of negative transfer as well as maximizes the multi-source positive transfer. We conduct extensive experiments under multi-source few-shot learning settings on real financial fraud detection dataset. Experimental analysis shows that the inter-domain relationships learned by our domain distance learning model align well with the facts and the results demonstrate that our multi-source transfer learning approach achieves significant improvements over the state-of-the-art transfer learning approaches.

The cognitive ability to learn with experience is a hallmark of intelligent systems. The emerging transfer optimization paradigm pursues such human-like problem-solving prowess by leveraging useful information from various source tasks to enhance optimization efficiency on a related target task. The occurrence of harmful negative transfer is a key concern in this setting, paving the way for recent probabilistic model-based transfer evolutionary algorithms that curb this phenomenon. However, in applications where the source and target domains, i.e., the features of their respective search spaces (e.g., dimensionality) and the distribution of good solutions in those spaces, do not match, narrow focus on curbing negative effects can lead to the conservative cancellation of knowledge transfer. Taking this cue, this paper presents a novel perspective on domain adaptation in the context of evolutionary optimization, inducing positive transfers even in scenarios of source-target domain mismatch. Our first contribution is to establish a probabilistic formulation of domain adaptation, by which source and/or target tasks can be mapped to a common solution representation space in which their discrepancy is reduced. Secondly, a domain adaptive transfer evolutionary algorithm is proposed, supporting both offline construction and online data-driven learning of non-linear mapping functions. The performance of the algorithm is experimentally verified, demonstrating superior convergence rate in comparison to state-of-the-art baselines on synthetic benchmarks and a practical case study in multi-location inventory planning. Our results thus shed light on a new research direction for optimization algorithms that improve their efficacy by learning from heterogeneous experiential priors. Agency for Science, Technology and Research (A*STAR) Nanyang Technological University This work was funded by the Agency for Science, Technology and Research (A*STAR) of Singapore, under the Singapore Institute of Manufacturing Technology-Nayang Technological University Collaborative Research Programme in Complex Systems. This work was also supported in part by the A*STAR Cyber-Physical Production System (CPPS) – Towards Contextual and Intelligent Response Research Program, under the RIE2020 IAF-PP Grant A19C1a0018. Yew-Soon Ong acknowledges the support by Singtel Cognitive and Artifcial Intelligence Lab for Enterprises (SCALE@ NTU), which is a collaboration between Singapore Telecommunications Limited (Singtel) and Nanyang Technological University (NTU) that is funded by the Singapore Government through the Industry Alignment Fund ‐ Industry Collaboration Projects Grant.

Recently, evolutionary multi-tasking (EMT) has been proposed as a new search paradigm for optimizing multiple problems simultaneously. Since the beneficial knowledge can be transferred among tasks to speed up the optimization process, EMT shows better performance in many problems compared with single-task evolutionary search algorithms. Notably, existing works on EMT have been devoted to the evolutionary search with two tasks. However, when multiple tasks (i.e., number of tasks >2) are involved, the existing methods might fail as it is necessary to decide which is the most suitable task to be selected for performing the knowledge transfer. To address this issue, we propose an online credit information based task selection algorithm to enhance the performance of EMT. Specifically, a credit matrix is introduced to express the transfer qualities among tasks. Then, we design two kinds of credit information and propose an updating mechanism to adjust the credit matrix online. After that, a greedy task selection mechanism is proposed for balancing the exploration and exploitation of the task selection process. Besides, we also propose an adaptive transfer rate to enhance positive transfer while reduce the impact of negative transfer. In the experiment, we compare our method with the existing works. The results clearly demonstrate the efficacy of the proposed method.

Domain adaptation is proposed to generalize learning machines and address performance degradation of models that are trained from one specific source domain but applied to novel target domains. Existing domain adaptation methods focus on transferring holistic features whose discriminability is generally tailored to be source-specific and inferiorly generic to be transferable. As a result, standard domain adaptation on holistic features usually damages feature structures, especially local feature statistics, and deteriorates the learned discriminability. To alleviate this issue, we propose to transfer primitive local feature patterns, whose discriminability are shown to be inherently more sharable, and perform hierarchical feature adaptation. Concretely, we first learn a cluster of domain-shared local feature patterns and partition the feature space into cells. Local features are adaptively aggregated inside each cell to obtain cell features, which are further integrated into holistic features. To achieve fine-grained adaptations, we simultaneously perform alignment on local features, cell features and holistic features, within which process the local and cell features are aligned independently inside each cell to maintain the learned local structures and prevent negative transfer. Experimenting on typical one-to-one unsupervised domain adaptation for both image classification and action recognition tasks, partial domain adaptation, and domain-agnostic adaptation, we show that the proposed method achieves more reliable feature transfer by consistently outperforming state-of-the-art models and the learned domain-invariant features generalize well to novel domains. Nanyang Technological University This work was supported by Zhejiang Lab (2020KB0AC02), National Key R&D Program of China (SQ2020YFB130047, 2020YFB1313501, 2020YFB1313503), Zhejiang Provincial Natural Science Foundation (LR19F020005), the National Natural Science Foundation of China (61972347, 31070944, 31271147, 61922019, 31471063, 31671074, and 61572433) and the Fundamental Research Funds for the Central Universities, China (2017FZA7003). Qian Zheng is supported by the Rapid-Rich Object Search (ROSE) Lab, Nanyang Technological University, Singapore.

Weakly-supervised learning has attracted growing research attention on medical lesions segmentation due to significant saving in pixel-level annotation cost. However, 1) most existing methods require effective prior and constraints to explore the intrinsic lesions characterization, which only generates incorrect and rough prediction; 2) they neglect the underlying semantic dependencies among weakly-labeled target enteroscopy diseases and fully-annotated source gastroscope lesions, while forcefully utilizing untransferable dependencies leads to the negative performance. To tackle above issues, we propose a new weakly-supervised lesions transfer framework, which can not only explore transferable domain-invariant knowledge across different datasets, but also prevent the negative transfer of untransferable representations. Specifically, a Wasserstein quantified transferability framework is developed to highlight widerange transferable contextual dependencies, while neglecting the irrelevant semantic characterizations. Moreover, a novel selfsupervised pseudo label generator is designed to equally provide confident pseudo pixel labels for both hard-to-transfer and easyto-transfer target samples. It inhibits the enormous deviation of false pseudo pixel labels under the self-supervision manner. Afterwards, dynamically-searched feature centroids are aligned to narrow category-wise distribution shift. Comprehensive theoretical analysis and experiments show the superiority of our model on the endoscopic dataset and several public datasets., Accepted to IEEE Transactions on Circuits and Systems for Video Technology (TCSVT 2020)

Domain adaptation aims to transfer knowledge across different domains and bridge the gap between them. While traditional knowledge transfer considers identical domain, a more realistic scenario is to transfer from a larger and more diverse source domain to a smaller target domain, which is referred to as partial domain adaptation (PDA). However, matching the whole source domain to the target domain for PDA might produce negative transfer. Samples in the shared classes should be carefully selected to mitigate negative transfer in PDA. We observe that the correlations between different target domain samples and source domain samples are diverse: classes are not equally correlated and moreover, different samples have different correlation strengthes even when they are in the same class. In this study, we propose ECDT, a novel PDA method that Exploits the Correlation Diversity for knowledge Transfer between different domains. We propose a novel method to estimate target domain label space that utilizes the label distribution and feature distribution of target samples, based on which outlier source classes can be filtered out and their negative effects on transfer can be mitigated. Moreover, ECDT combines class-level correlation and instance-level correlation to quantity sample-level transferability in domain adversarial network. Experimental results on three commonly used cross-domain object data sets show that ECDT is superior to previous partial domain adaptation methods.

Background One criterion of adaptive learning is appropriate generalization to new instances based on the original learning context and avoiding overgeneralization. Appropriate generalization requires understanding what features of a solution are applicable in a new context and whether the new context requires modifications or a new approach. In a series of three experiments, we investigate whether searching for an algebraic formalism before receiving direct instruction facilitates appropriate generalization. Results (1) Searching buffers against negative transfer: participants who first searched for an equation were less likely to overgeneralize compared to participants who completed a tell-and-practice activity. (2) Likelihood of creating a correct new adaptation varied by performance on the searching task. (3) Asking people to sketch alleviated some of the negative effects of tell-and-practice, but sketching did not augment the effect of searching. (4) When participants received more elaborate tell-and-practice instruction, the advantages of searching were less notable. Conclusions Searching for an algebraic formula prior to direct instruction may be a productive way to help learners connect a formula to its referent and avoid overgeneralization. Tell-and-practice instruction that only described the mathematical procedures led to the greatest levels of overgeneralization errors and worst performance. Tell-and-practice instruction that highlighted connections between the mathematical structure of the formula and the visual referent performed at similar or marginally worse levels than the search-first conditions.

Cross-dataset facial expression recognition (FER) has remained a challenging problem due to the obvious biases caused by diverse subjects and various collection conditions. To this end, domain adaption can be adopted as an effective solution by learning invariant representations across domains (datasets). However, FER requires special consideration of its specific problems e.g., uncertainties caused by ambiguous facial images, and diverse inter- and intra-class relationship. Such uncertainties already exist in single dataset FER, and could be significantly aggravated by enlarged class-wise discrepancies under cross-dataset scenarios. To mitigate this problem, this paper proposes an unsupervised domain adaptation method via regularized conditional alignment for FER, which adversarially reduces domain- and class-wise discrepancies while explicitly dealing with uncertainties within and across domain. Specifically, the proposed method effectively suppresses uncertainties in FER transfer tasks via: 1) semantics-preserving adaptation framework which enforces both domain-invariant learning and class-level semantic consistency between source and target expression data, where discriminative cluster structures are simultaneously retained; 2) auxiliary uncertainty regularization which further constrains the ambiguity of cluster boundaries to guarantee the transferring reliability, thus discouraging the negative transfer brought by divergent facial images. Evaluation experiments on publicly available datasets demonstrate that the proposed method significantly outperforms the current state-of-the-art methods.

Rapid Serial Visual Presentation (RSVP)-based Brain-Computer Interface (BCI) is an efficient information detection technology by detecting event-related brain responses evoked by target visual stimuli. However, a time-consuming calibration procedure is needed before a new user can use this system. Thus, it is important to reduce calibration efforts for BCI applications. In this paper, we collect an RSVP-based electroencephalogram (EEG) dataset, which includes 11 subjects. The experimental task is image retrieval. Also, we propose a multi-source transfer learning framework by utilizing data from other subjects to reduce the data requirement on the new subject for training the model. A source-selection strategy is firstly adopted to avoid negative transfer. And then, we propose a transfer learning network based on domain adversarial training. The convolutional neural network (CNN)-based network is designed to extract common features of EEG data from different subjects, while the discriminator tries to distinguish features from different subjects. In addition, a classifier is added for learning semantic information. Also, conditional information and gradient penalty are added to enable stable training of the adversarial network and improve performance. The experimental results demonstrate that our proposed method outperforms a series of state-of-the-art and baseline approaches.

Transfer learning uses knowledge learnt in source domains to aid predictions in a target domain. When source and target domains are online, they are susceptible to concept drift, which may alter the mapping of knowledge between them. Drifts in online environments can make additional information available in each domain, necessitating continuing knowledge transfer both from source to target and vice versa. To address this, we introduce the Bi-directional Online Transfer Learning (BOTL) framework, which uses knowledge learnt in each online domain to aid predictions in others. We introduce two variants of BOTL that incorporate model culling to minimise negative transfer in frameworks with high volumes of model transfer. We consider the theoretical loss of BOTL, which indicates that BOTL achieves a loss no worse than the underlying concept drift detection algorithm. We evaluate BOTL using two existing concept drift detection algorithms: RePro and ADWIN. Additionally, we present a concept drift detection algorithm, Adaptive Windowing with Proactive drift detection (AWPro), which reduces the computation and communication demands of BOTL. Empirical results are presented using two data stream generators: the drifting hyperplane emulator and the smart home heating simulator, and real-world data predicting Time To Collision (TTC) from vehicle telemetry. The evaluation shows BOTL and its variants outperform the concept drift detection strategies and the existing state-of-the-art online transfer learning technique.

Multi-task learning (MTL) has been successfully applied to many recommendation applications. However, MTL models often suffer from performance degeneration with negative transfer due to the complex and competing task correlation in real-world recommender systems. Moreover, through extensive experiments across SOTA MTL models, we have observed an interesting seesaw phenomenon that performance of one task is often improved by hurting the performance of some other tasks. To address these issues, we propose a Progressive Layered Extraction (PLE) model with a novel sharing structure design. PLE separates shared components and task-specific components explicitly and adopts a progressive routing mechanism to extract and separate deeper semantic knowledge gradually, improving efficiency of joint representation learning and information routing across tasks in a general setup. We apply PLE to both complicatedly correlated and normally correlated tasks, ranging from two-task cases to multi-task cases on a real-world Tencent video recommendation dataset with 1 billion samples, and results show that PLE outperforms state-of-the-art MTL models significantly under different task correlations and task-group size. Furthermore, online evaluation of PLE on a large-scale content recommendation platform at Tencent manifests 2.23% increase in view-count and 1.84% increase in watch time compared to SOTA MTL models, which is a significant improvement and demonstrates the effectiveness of PLE. Finally, extensive offline experiments on public benchmark datasets demonstrate that PLE can be applied to a variety of scenarios besides recommendations to eliminate the seesaw phenomenon. PLE now has been deployed to the online video recommender system in Tencent successfully.

With the deployment of smart homes, we find that human activity recognition (HAR) is essentially important to many applications, e.g., child/senior care, intelligent information push and exercise promotion. Although it is always better to build HAR model for each smart home to resolve the practical problem that homes have different floorplans or adopted sensors, it is intractable to acquire labeled data for each home due to cost and privacy. We thus propose a method to transfer the HAR model from multiple labeled source homes to the unlabeled target home. Specifically, we first generate transferable representations for the sensors of these homes, based on which we build the HAR model using the data of labeled source homes. Then, we employ the built HAR model into the unlabeled target home. Experiment results on CASAS dataset illustrate that our proposed method outperforms baseline methods in general and also avoids potential negative transfer caused by using only one source home.

Multi-task learning (MTL) aims to make full use of the knowledge contained in multi-task supervision signals to improve the overall performance. How to make the knowledge of multiple tasks shared appropriately is an open problem for MTL. Most existing deep MTL models are based on parameter sharing. However, suitable sharing mechanism is hard to design as the relationship among tasks is complicated. In this paper, we propose a general framework called Multi-Task Neural Architecture Search (MTNAS) to efficiently find a suitable sharing route for a given MTL problem. MTNAS modularizes the sharing part into multiple layers of sub-networks. It allows sparse connection among these sub-networks and soft sharing based on gating is enabled for a certain route. Benefiting from such setting, each candidate architecture in our search space defines a dynamic sparse sharing route which is more flexible compared with full-sharing in previous approaches. We show that existing typical sharing approaches are sub-graphs in our search space. Extensive experiments on three real-world recommendation datasets demonstrate MTANS achieves consistent improvement compared with single-task models and typical multi-task methods while maintaining high computation efficiency. Furthermore, in-depth experiments demonstrates that MTNAS can learn suitable sparse route to mitigate negative transfer.