Search

Deep reinforcement learning is confronted with problems of sampling inefficiency and poor task migration capability. Meta-reinforcement learning (meta-RL) enables meta-learners to utilize the task-solving skills trained on similar tasks and quickly adapt to new tasks. However, meta-RL methods lack enough queries toward the relationship between task-agnostic exploitation of data and task-related knowledge introduced by latent context, limiting their effectiveness and generalization ability. In this article, we develop an algorithm for off-policy meta-RL that can provide the meta-learners with self-oriented cognition toward how they adapt to the family of tasks. In our approach, we perform dynamic task-adaptiveness distillation to describe how the meta-learners adjust the exploration strategy in the meta-training process. Our approach also enables the meta-learners to balance the influence of task-agnostic self-oriented adaption and task-related information through latent context reorganization. In our experiments, our method achieves 10%-20% higher asymptotic reward than probabilistic embeddings for actor-critic RL (PEARL).

Biped robots with dynamic motion control have shown strong robustness in complex environments. However, many motion planning methods rely on models, which have difficulty dynamically modifying the walking cycle, height, and other gait parameters to cope with environmental changes. In this study, a heuristic model-free gait template planning method with dynamic motion control is proposed. The gait trajectory can be generated by inputting the desired speed, walking cycle, and support height without a model. Then, the stable walking of the biped robot can be realized by foothold adjustment and whole-body dynamics model control. The gait template can be changed in real time to achieve gait flexibility of the biped robot. Finally, the effectiveness of the method is verified by simulations and experiments of the biped robot BHR-B2. The research presented here helps improve the gait transition ability of biped robots in dynamic locomotion.

Dynamic neural network is an emerging research topic in deep learning. Compared to static models which have fixed computational graphs and parameters at the inference stage, dynamic networks can adapt their structures or parameters to different inputs, leading to notable advantages in terms of accuracy, computational efficiency, adaptiveness, etc. In this survey, we comprehensively review this rapidly developing area by dividing dynamic networks into three main categories: 1) instance-wise dynamic models that process each instance with data-dependent architectures or parameters; 2) spatial-wise dynamic networks that conduct adaptive computation with respect to different spatial locations of image data and 3) temporal-wise dynamic models that perform adaptive inference along the temporal dimension for sequential data such as videos and texts. The important research problems of dynamic networks, e.g., architecture design, decision making scheme, optimization technique and applications, are reviewed systematically. Finally, we discuss the open problems in this field together with interesting future research directions.

Reinforcement learning (RL) is a promising technique for designing a model-free controller by interacting with the environment. Several researchers have applied RL to autonomous underwater vehicles (AUVs) for motion control, such as trajectory tracking. However, the existing RL-based controller usually assumes that the unknown AUV dynamics keep invariant during the operation period, limiting its further application in the complex underwater environment. In this article, a novel meta-RL-based control scheme is proposed for trajectory tracking control of AUV in the presence of unknown and time-varying dynamics. To this end, we divide the tracking task for AUV with time-varying dynamics into multiple specific tasks with fixed time-varying dynamics, to which we apply meta-RL for training to distill the general control policy. The obtained control policy can transfer to the testing phase with high adaptability. Inspired by the line-of-sight (LOS) tracking rule, we formulate each specific task as a Markov decision process (MDP) with a well-designed state and reward function. Furthermore, a novel policy network with an attention module is proposed to extract the hidden information of AUV dynamics. The simulation environment with time-varying dynamics is established, and the simulation results reveal the effectiveness of our proposed method.

Smile analysis in horizontally angled views is indispensable for esthetic assessment and could reveal teeth-to-lip disharmony, which might escape discovery in the frontal and profile views. However, evidence is lacking on where the anterior teeth should be positioned for esthetics in angled smiles.Based on 3-dimensional facial image processing and geometric analysis, the lip edges were projected to the horizontal plane, and the horizontal teeth-to-lip relation was simplified and represented by the distances from dental landmarks to lower bow-shaped curves (LBSC), with the distance from facial-axis (FA) point of the canine to LBSC (FA-tangent line [TL] distance) identified as the key parameter. Using photographic modification and esthetic assessment, the effect of FA-TL distances on the attractiveness of 45° angled smiles was tested, with esthetic ranges identified. A simplified method was developed to obtain the estimative LBSC and FA-TL distances using 2-dimensional photographs and geometric analysis to facilitate clinical application.The FA-TL distance remarkably affected the esthetics of 45° angled smiles. Smiles were attractive when the FA-TL distance ranged from -1.0 to 1.5 mm perceived by orthodontists and from -1.5 to 1.5 mm perceived by laypersons. The 2-dimensional photograph-derived estimative FA-TL distance was not significantly different from that obtained in a 3-dimensional image, validating the simplified method.The LBSC could serve as a reference frame to determine the lateral limit of the maxillary anterior arch for the esthetics of 45° angled smiles. The FA-TL distance, which represented the spatial relation of the maxillary canine with the lower lip, was an esthetically essential parameter. For females aged 20-30 years, the FA point of the maxillary canines should be positioned no more than 1.5 mm labial or lingual to the LBSC.

IntroductionMalignant pancreatic cancer has poor long-term survival. Increasing evidence shows that FAM83A (family with sequence similarity 83 member A) plays a vital role in tumorigenesis and malignant progression in some human cancer types. The present study explored the potential mechanism of FAM83A in improving the prognosis of pancreatic cancer patients.MethodsTranscriptomic and clinical data from patients were obtained from The Cancer Genome Atlas while FAM83A expression was measured in tumorous pancreatic tissue compared with normal controls by quantitative real-time PCR and immunohistochemistry.ResultsFAM83A is a vital prognostic indicator and potential oncogene in pancreatic cancer via pan-cancer analysis. In silico analysis revealed that AL049555.1/hsa-miR-129-5p axis was the pivotal upstream ncRNA- mediated pathway of FAM83A in pancreatic cancer. Furthermore, FAM83A expression was related to immune cell infiltration through vital immune-related genes including programmed cell death 1 (PDCD1), and tumorigenesis through common mutation genes including KRAS protooncogene GTPase (KRAS), and SMAD family member 4 (SMAD4). In summary, ncRNA-mediated upregulation of FAM83A is associated with poor long-term survival and immune cell infiltration in pancreatic cancer.DiscussionFAM83A may be used as a novel survival-related and immune-related biomarker. This information suggests that FAM83A may be a novel therapeutic target for combined or individual treatment for patients with pancreatic cancer.

The synthesis of cyano-containing pyrrolo- and piperidino-quinazolinones was developed using alkyl nitriles through radical cascade addition/cyclization under metal-free conditions.

In many real-world scenarios, extrinsic rewards provided by the environment are sparse. An agent trained with classic reinforcement learning algorithm fails to explore these environments in a sufficient and effective way. To address this problem, the exploration bonus which derives from environmental novelty serves as intrinsic motivation for the agent. In recent years, curiosity-driven exploration is a mainstream approach to describe environmental novelty through prediction errors of dynamics models. Due to the expressive ability limitations of curiosity-based environmental novelty and the difficulty of finding appropriate feature space, most curiosity-driven exploration methods have the problem of overprotection against repetition. This problem can reduce the efficiency of exploration and lead the agent into a trap with local optimality. In this article, we propose a combination of persisting curiosity and temporary curiosity framework to deal with the problem of overprotection against repetition. We introduce the self-attention mechanism from the field of computer vision and propose a sequence-based self-attention mechanism for temporary curiosity generation. We compare our framework with some previous exploration methods in hard-exploration environments, provide a series of comprehensive analysis of the proposed framework and investigate the effect of the individual components of our method. The experimental results indicate that the proposed framework delivers superior performance than existing methods.

The mechanical and thermodynamic properties of Al2Ca and Mg2Ca in the pressure range of 0~100 Gpa were investigated using first-principles calculations. The structural parameters, such as lattice constant ratio, unit cell volume ratio, density, were investigated. The calculated elastic constants satisfy the born’s stability criterion, indicating that they are mechanically stable at normal and high pressure. Mechanical parameters such as bulk modulus, shear modulus, and Young’s modulus of polycrystalline materials have been derived from single-crystal elastic constants. The Poisson’s ratio and anisotropy were investigated. The results show that the B/G value of Mg2Ca is greater than 1.75, indicating it is a ductile phase under various pressures. When the pressure was equal to 40 Gpa, Al2Ca was transferred brittle to toughness, and the bulk modulus, shear modulus, and Young’s modulus of Al2Ca were all larger than those of Mg2Ca, indicating that the comprehensive mechanical properties of Al2Ca are better than those of Mg2Ca. The constant heat capacity obtained by the quasi-harmonic approximation indicates that the ability of Mg2Ca to release or store heat is greater than that of Al2Ca. Moreover, the coefficient of thermal expansion (α) increases exponentially at lower temperatures and linearly at higher temperatures for both alloys.

Background Accumulating articles demonstrate that circular RNAs play pivotal functions in tumorigenesis. However, the working mechanism of circ_0136666 in osteosarcoma (OS) progression remains to be further clarified. Methods Real time-quantitative polymerase chain reaction and western blot assay were applied to determine RNA and protein expression, respectively. Cell proliferation was assessed by 5-Ethynyl-2′-deoxyuridine assay and colony formation assay. Transwell assays were carried out to assess cell migration and invasion abilities. Flow cytometry was performed to analyze cell apoptosis. Cell glycolysis was evaluated by analyzing the uptake of glucose and the production of lactate using the corresponding kits. Dual-luciferase reporter assay and biotinylated RNA-pull down assay were performed to confirm the target interaction between microRNA-1244 (miR-1244) and circ_0136666 or centrosomal protein 55 (CEP55). Xenograft tumor model was utilized to explore the role of circ_0136666 in tumor growth in vivo. Results Circ_0136666 expression was prominently elevated in OS tissues and cell lines. Circ_0136666 absence restrained the proliferation, migration, invasion and glycolytic metabolism and promoted the apoptosis of OS cells. Circ_0136666 negatively regulated miR-1244 expression by binding to it in OS cells. MiR-1244 overexpression suppressed the malignant behaviors of OS cells. CEP55 was a target of miR-1244 in OS cells. Circ_0136666 positively regulated CEP55 expression partly by sequestering miR-1244 in OS cells. CEP55 overexpression largely reversed circ_0136666 silencing-mediated influences in OS cells. Circ_0136666 silencing significantly suppressed tumor growth in vivo. Conclusion Circ_0136666 silencing inhibited OS progression partly by targeting miR-1244/CEP55 signaling. Silencing circ_0136666 and CEP55 or restoring miR-1244 level might be a potential therapeutic strategy for OS.

A rhodium(III)-catalyzed C6-selective dehydrogenative cross-coupling of 2-pyridones with thiophenes was developed for the synthesis of 6-thiophenyl pyridin-2(1H)-one derivatives. In this reaction, the excellent site selectivity was controlled by the 2-pyridyl directing group on the nitrogen of the pyridone ring. Control experiments indicated that the N-pyridyl was essential for the transformation. To the best of our knowledge, this procedure is the first successful example of the direct C6 heteroarylation of 2-pyridones with electron-rich thiophene derivatives. 4-Pyridone was also used as substrate to generate the corresponding C2 heteroarylated product. Moreover, this pyridyl directing group was readily removable to generate the biheteroaryl structures with a free N-H group.

In this study, a rapid and sensitive analytical method has been developed to detect tetracycline hydrochloride (TC) using manganese-doped carbon dots (Mn-CDs) prepared by one-step hydrothermal procedure using 1-(2-pyridinylazo)-2-naohthalenol (PAN) and MnCl2 as precursor reagents. The obtained Mn-CDs showed an ultraviolet emission at 360 nm with an excitation wavelength of 300 nm. TC has a strong characteristic absorption peak at 356 nm, which has a large spectral overlap with the emission band of the Mn-CDs. The fluorescence intensity (FI) of Mn-CDs at 360 nm is linearly quenched within the TC concentration range of 0.1-200 µM. The developed assay for the detection of TC was based on an inner filter effect (IFE) mechanism and is rapid, sensitive, and was successfully applied for the determination of TC in different poultry meat samples with satisfactory results.

Dimension reduction (DR)-based on extreme learning machine auto-encoder (ELM-AE) has achieved many successes in recent years. By minimizing the self-reconstruction error, the ELM-AE-based DR algorithms learn the compressed representations which facilitate the subsequent classification. However, the existing ELM-AEs only consider the DR problem in an unsupervised manner and ignore the valuable supervised information when these information is available. To find discriminative features of the original data, in this paper, we propose a graph embedding-based DR framework with ELM (GDR-ELM) for DR problems. Instead of self-reconstruction, the proposed GDR-ELM reconstructs all samples according to the weights in a graph matrix containing the supervised information. Furthermore, GDR-ELM can be stacked as building blocks to construct a multilayer framework like other ELM-AEs for more complicated representation learning tasks. Experiments on various datasets demonstrate the effectiveness of the proposed GDR-ELM and its multilayer framework.

This paper first introduces the guide-weight criterion into the topology optimization problems for maximization of the fundamental eigenfrequency of vibrating continuum structures. The traditional solid isotropic material with penalization model is modified to eliminate the artificial localized modes. Based on this modified model, the iteration formula of the design variables is derived using the guide-weight criterion. An iterative mass control strategy is adopted to satisfy the equality constraint on the final mass and to stabilize the iteration process. Additionally, a mass preserving density filter based on Heaviside function is used to solve the gray transition problem. Several typical examples are used to validate the proposed method. Numerical results show that the proposed method is capable of achieving iterative convergence and clear profiles of topologies; meanwhile, the optimal results obtained by the proposed method agree well with those obtained by the commonly used bi-directional evolutionary structural optimization (BESO) method. In particular, the proposed method has a faster convergence rate than the BESO method.

Image-text matching aims to find the relationship between image and text data and to establish a connection between them. The main challenge of image-text matching is the fact that images and texts have different data distributions and feature representations. Current methods for image-text matching fall into two basic types: methods that map image and text data into a common space and then use distance measurements and methods that treat image-text matching as a classification problem. In both cases, the two data modes used are image and text data. In our method, we create a fusion layer to extract intermediate modes, thus improving the image-text processing results. We also propose a concise way to update the loss function that makes it easier for neural networks to handle difficult problems. The proposed method was verified on the Flickr30K and MS-COCO datasets and achieved superior matching results compared to existing methods.

BACKGROUND Strategies for integrating artificial intelligence (AI) into thyroid nodule management require additional development and testing. We developed a deep-learning AI model (ThyNet) to differentiate between malignant tumours and benign thyroid nodules and aimed to investigate how ThyNet could help radiologists improve diagnostic performance and avoid unnecessary fine needle aspiration. METHODS ThyNet was developed and trained on 18 049 images of 8339 patients (training set) from two hospitals (the First Affiliated Hospital of Sun Yat-sen University, Guangzhou, China, and Sun Yat-sen University Cancer Center, Guangzhou, China) and tested on 4305 images of 2775 patients (total test set) from seven hospitals (the First Affiliated Hospital of Guangzhou University of Chinese Medicine, Guangzhou, China; the Sixth Affiliated Hospital of Sun Yat-sen University, Guangzhou, China; the Guangzhou Army General Hospital, Guangzhou, China; the Third Affiliated Hospital of Sun Yat-sen University, Guangzhou, China; the First Affiliated Hospital of Sun Yat-sen University; Sun Yat-sen University Cancer Center; and the First Affiliated Hospital of Guangxi Medical University, Nanning, China) in three stages. All nodules in the training and total test set were pathologically confirmed. The diagnostic performance of ThyNet was first compared with 12 radiologists (test set A); a ThyNet-assisted strategy, in which ThyNet assisted diagnoses made by radiologists, was developed to improve diagnostic performance of radiologists using images (test set B); the ThyNet assisted strategy was then tested in a real-world clinical setting (using images and videos; test set C). In a simulated scenario, the number of unnecessary fine needle aspirations avoided by ThyNet-assisted strategy was calculated. FINDINGS The area under the receiver operating characteristic curve (AUROC) for accurate diagnosis of ThyNet (0·922 [95% CI 0·910-0·934]) was significantly higher than that of the radiologists (0·839 [0·834-0·844]; p

Objectives To evaluate the feasibility and advantages of wedge resection plus transverse suture without mesentery detached approach applied to loop ileostomy closure by analyzing the surgical data and the incidence of postoperative complications of patients undergoing this procedure. Methods We performed a retrospective analysis of the hospitalization data of patients who underwent ileostomy closure surgery and met the research standards from January 2017 to April 2021 in Guangxi Medical University Cancer Hospital; all surgeries were performed by the same surgeon. The perioperative data were statistically analyzed by grouping. Results In total, 65 patients were enrolled in this study, with 12 in the wedge resection group, 35 in the stapler group, and 18 in the hand suture group. There was no significant difference in operation time between the wedge resection group and stapler group (P > 0.05), but both groups had shorter operation time than that in the hand suture group (P Conclusions The wedge resection plus transverse suture without mesentery detached approach is safe and easy for closure of loop ileostomy in selected patients, and the intestinal motility recovers rapidly postoperatively. It costs less surgical consumables, and is particularly suitable for the currently implemented Diagnosis-Related Groups payment method.

To compare the differences in peripheral blood inflammatory indices between patients with neovascular age-related macular degeneration (nAMD) and haemorrhagic polypoidal choroidal vasculopathy (PCV).Retrospective, best corrected visual acuity (BCVA), neutrophil-to-lymphocyte ratio (NLR), platelet-to-lymphocyte ratio (PLR) and monocyte-to-lymphocyte ratio (MLR), were analysed across the nAMD, PCV and normal control (NCG) groups of patients. The ratios' cut-off values for nAMD were calculated.nAMD had a significantly longer duration and better BCVA than PCV (all P.05). The NLR, MLR and PLR were significantly higher in nAMD than in PCV and NCG (all P.01), no significant differences between PCV and NCG (all P.05). The ROC curve analysis revealed that the cut-off values for NLR and MLR were 1.98 and 0.24, respectively, for nAMD.NLR, MLR and PLR are significantly high in patients with nAMD. The ability of these inflammatory indicators to distinguish nAMD and PCV is unclear.

Efficient and collision-free coordination of two robot arms is increasingly needed in various service-oriented robotic applications. This paper proposes a dual arm coordination algorithm to improve the efficiency of coordination by considering both robot’s actions and operating sequences of the tasks that need to use two arms to complete complex operations. Teleoperation demonstration is first performed to obtain the robot’s human-like motion trajectories, so as to reduce the probability of the collisions between the two arms. The coordination diagram in time domain is then designed to more clearly represent the situations of trajectory collisions and find the collision-free coordination action law. A Coordination Pair Generator (CPG) is designed to reorganize the operating sequences according to the characteristics of input trajectories and the action coordination. The effectiveness and efficiency of our method are verified on the simulation and physical experiments which execute the drug sorting task in nursing homes, respectively, on the ABB YuMi robot model and self-developed robot system. According to the experiment results, the operation time has been reduced by 9% and the collision area has been reduced by 7.5%.

Meta reinforcement learning (meta-RL) is a promising technique for fast task adaptation by leveraging prior knowledge from previous tasks. Recently, context-based meta-RL has been proposed to improve data efficiency by applying a principled framework, dividing the learning procedure into task inference and task execution. However, the task information is not adequately leveraged in this approach, thus leading to inefficient exploration. To address this problem, we propose a novel context-based meta-RL framework with an improved exploration mechanism. For the existing exploration and execution problem in context-based meta-RL, we propose a novel objective that employs two exploration terms to encourage better exploration in action and task embedding space, respectively. The first term pushes for improving the diversity of task inference, while the second term, named action information, works as sharing or hiding task information in different exploration stages. We divide the meta-training procedure into task-independent exploration and task-relevant exploration stages according to the utilization of action information. By decoupling task inference and task execution and proposing the respective optimization objectives in the two exploration stages, we can efficiently learn policy and task inference networks. We compare our algorithm with several popular meta-RL methods on MuJoco benchmarks with both dense and sparse reward settings. The empirical results show that our method significantly outperforms baselines on the benchmarks in terms of sample efficiency and task performance.

Spatial redundancy commonly exists in the learned representations of convolutional neural networks (CNNs), leading to unnecessary computation on high-resolution features. In this paper, we propose a novel Spatially Adaptive feature Refinement (SAR) approach to reduce such superfluous computation. It performs efficient inference by adaptively fusing information from two branches: one conducts standard convolution on input features at a lower spatial resolution , and the other one selectively refines a set of regions at the original resolution. The two branches complement each other in feature learning, and both of them evoke much less computation than standard convolution. SAR is a flexible method that can be conveniently plugged into existing CNNs to establish models with reduced spatial redundancy. Experiments on CIFAR and ImageNet classification, COCO object detection and PASCAL VOC semantic segmentation tasks validate that the proposed SAR can consistently improve the network performance and efficiency. Notably, our results show that SAR only refines less than 40% of the regions in the feature representations of a ResNet for 97% of the samples in the validation set of ImageNet to achieve comparable accuracy with the original model, revealing the high computational redundancy in the spatial dimension of CNNs.

Action recognition is an important and challenging problem in video analysis. Although the past decade has witnessed progress in action recognition with the development of deep learning, such process has been slow in competitive sports content analysis. To promote the research on action recognition from competitive sports video clips, we introduce a Figure Skating Dataset (FSD-10) for fine-grained sports content analysis. To this end, we collect 1484 clips from the worldwide figure skating championships in 2017–2018, which consist of 10 different actions in men/ladies programs. Each clip is at a rate of 30 frames per second with resolution 1080 × 720, which are annotated by experts. To build a baseline for action recognition in figure skating, we evaluate state-of-the-art action recognition methods on FSD-10. Motivated by the idea that domain knowledge is of great concern in sports field, we propose a keyframe based temporal segment network (KTSN) for classification and achieve remarkable performance. Experimental results demonstrate that FSD-10 is an ideal dataset for benchmarking action recognition algorithms, as it requires to accurately extract action motions rather than action poses. We hope FSD-10, which is designed to have a large collection of finegrained actions, can serve as a new challenge to develop more robust and advanced action recognition models.

Autophagy has been shown to maintain neural system homeostasis during stroke. However, the molecular mechanisms underlying neuronal autophagy in ischemic stroke remain poorly understood. This study aims to investigate the regulatory mechanisms of the pathway consisting of MEG3 (maternally expressed gene 3), microRNA-378 (miR-378), and GRB2 (growth factor receptor-bound protein 2) in neuronal autophagy and neurological functional impairment in ischemic stroke. A mouse model of the middle cerebral artery occluded–induced ischemic stroke and an in vitro model of oxygen-glucose deprivation–induced neuronal injury were developed. To understand the role of the MEG3/miR-378/GRB2 axis in the neuronal regulation, the expression of proteins associated with autophagy in neurons was measured by Western blotting analysis, and neuron death was evaluated using a lactate dehydrogenase leakage rate test. First, it was found that the GRB2 gene, up-regulated in middle cerebral artery occluded–operated mice and oxygen-glucose deprivation–exposed neurons, was a target gene of miR-378. Next, miR-378 inhibited neuronal loss and neurological functional impairment in mice, as well as neuronal autophagy and neuronal death by silencing of GRB2. Confirmatory in vitro experiments showed that MEG3 could specifically bind to miR-378 and subsequently up-regulate the expression of GRB2, which in turn suppressed the activation of Akt/mTOR pathway. Taken together, these findings suggested that miR-378 might protect against neuronal autophagy and neurological functional impairment and proposed that a MEG3/miR-378/GRB2 regulatory axis contributed to better understanding of the pathophysiology of ischemic stroke.

Tremendous research efforts have been made to thrive deep domain adaptation (DA) by seeking domain-invariant features. Most existing deep DA models only focus on aligning feature representations of task-specific layers across domains while integrating a totally shared convolutional architecture for source and target. However, we argue that such strongly-shared convolutional layers might be harmful for domain-specific feature learning when source and target data distribution differs to a large extent. In this paper, we relax a shared-convnets assumption made by previous DA methods and propose a Domain Conditioned Adaptation Network (DCAN), which aims to excite distinct convolutional channels with a domain conditioned channel attention mechanism. As a result, the critical low-level domain-dependent knowledge could be explored appropriately. As far as we know, this is the first work to explore the domain-wise convolutional channel activation for deep DA networks. Moreover, to effectively align high-level feature distributions across two domains, we further deploy domain conditioned feature correction blocks after task-specific layers, which will explicitly correct the domain discrepancy. Extensive experiments on three cross-domain benchmarks demonstrate the proposed approach outperforms existing methods by a large margin, especially on very tough cross-domain learning tasks., Accepted by AAAI 2020

Attention-based neural networks, such as Transformers, have become ubiquitous in numerous applications, including computer vision, natural language processing, and time-series analysis. In all kinds of attention networks, the attention maps are crucial as they encode semantic dependencies between input tokens. However, most existing attention networks perform modeling or reasoning based on representations , wherein the attention maps of different layers are learned separately without explicit interactions. In this paper, we propose a novel and generic evolving attention mechanism, which directly models the evolution of inter-token relationships through a chain of residual convolutional modules. The major motivations are twofold. On the one hand, the attention maps in different layers share transferable knowledge, thus adding a residual connection can facilitate the information flow of inter-token relationships across layers. On the other hand, there is naturally an evolutionary trend among attention maps at different abstraction levels, so it is beneficial to exploit a dedicated convolution-based module to capture this process. Equipped with the proposed mechanism, the convolution-enhanced evolving attention networks achieve superior performance in various applications, including time-series representation, natural language understanding, machine translation, and image classification. Especially on time-series representation tasks, Evolving Attention-enhanced Dilated Convolutional (EA-DC-) Transformer outperforms state-of-the-art models significantly, achieving an average of 17% improvement compared to the best SOTA. To the best of our knowledge, this is the first work that explicitly models the layer-wise evolution of attention maps. Our implementation is available at https://github.com/pkuyym/EvolvingAttention., Comment: Accepted by IEEE T-PAMI. Extension of the previous work (arXiv:2102.12895). arXiv admin note: text overlap with arXiv:2102.12895

Traditional knowledge distillation transfers "dark knowledge" of a pre-trained teacher network to a student network, and ignores the knowledge in the training process of the teacher, which we call teacher's experience. However, in realistic educational scenarios, learning experience is often more important than learning results. In this work, we propose a novel knowledge distillation method by integrating the teacher's experience for knowledge transfer, named experience ensemble knowledge distillation (EEKD). We save a moderate number of intermediate models from the training process of the teacher model uniformly, and then integrate the knowledge of these intermediate models by ensemble technique. A self-attention module is used to adaptively assign weights to different intermediate models in the process of knowledge transfer. Three principles of constructing EEKD on the quality, weights and number of intermediate models are explored. A surprising conclusion is found that strong ensemble teachers do not necessarily produce strong students. The experimental results on CIFAR-100 and ImageNet show that EEKD outperforms the mainstream knowledge distillation methods and achieves the state-of-the-art. In particular, EEKD even surpasses the standard ensemble distillation on the premise of saving training cost., Comment: Under review