Search

Deep reinforcement learning is confronted with problems of sampling inefficiency and poor task migration capability. Meta-reinforcement learning (meta-RL) enables meta-learners to utilize the task-solving skills trained on similar tasks and quickly adapt to new tasks. However, meta-RL methods lack enough queries toward the relationship between task-agnostic exploitation of data and task-related knowledge introduced by latent context, limiting their effectiveness and generalization ability. In this article, we develop an algorithm for off-policy meta-RL that can provide the meta-learners with self-oriented cognition toward how they adapt to the family of tasks. In our approach, we perform dynamic task-adaptiveness distillation to describe how the meta-learners adjust the exploration strategy in the meta-training process. Our approach also enables the meta-learners to balance the influence of task-agnostic self-oriented adaption and task-related information through latent context reorganization. In our experiments, our method achieves 10%-20% higher asymptotic reward than probabilistic embeddings for actor-critic RL (PEARL).

Dynamic neural network is an emerging research topic in deep learning. Compared to static models which have fixed computational graphs and parameters at the inference stage, dynamic networks can adapt their structures or parameters to different inputs, leading to notable advantages in terms of accuracy, computational efficiency, adaptiveness, etc. In this survey, we comprehensively review this rapidly developing area by dividing dynamic networks into three main categories: 1) instance-wise dynamic models that process each instance with data-dependent architectures or parameters; 2) spatial-wise dynamic networks that conduct adaptive computation with respect to different spatial locations of image data and 3) temporal-wise dynamic models that perform adaptive inference along the temporal dimension for sequential data such as videos and texts. The important research problems of dynamic networks, e.g., architecture design, decision making scheme, optimization technique and applications, are reviewed systematically. Finally, we discuss the open problems in this field together with interesting future research directions.

Reinforcement learning (RL) is a promising technique for designing a model-free controller by interacting with the environment. Several researchers have applied RL to autonomous underwater vehicles (AUVs) for motion control, such as trajectory tracking. However, the existing RL-based controller usually assumes that the unknown AUV dynamics keep invariant during the operation period, limiting its further application in the complex underwater environment. In this article, a novel meta-RL-based control scheme is proposed for trajectory tracking control of AUV in the presence of unknown and time-varying dynamics. To this end, we divide the tracking task for AUV with time-varying dynamics into multiple specific tasks with fixed time-varying dynamics, to which we apply meta-RL for training to distill the general control policy. The obtained control policy can transfer to the testing phase with high adaptability. Inspired by the line-of-sight (LOS) tracking rule, we formulate each specific task as a Markov decision process (MDP) with a well-designed state and reward function. Furthermore, a novel policy network with an attention module is proposed to extract the hidden information of AUV dynamics. The simulation environment with time-varying dynamics is established, and the simulation results reveal the effectiveness of our proposed method.

Objective: To observe the changes of diagnosis and treatment of a patient with rhegmatogenous retinal detachment (RRD) after phakic intraocular lens implantation in high myopia, and analyze its mechanism and treatment combined with the literature. Methods: To report a case of rhegmatogenous retinal detachment (RRD) after phakic intraocular lens implantation. Result: Two months after the patient underwent scleral cerclage + pad pressing + vitrectomy + silicone oil filling, the fundus color photos showed that the vitreous cavity was filled with silicone oil, the fundus retina was flat, the cerclage ridge was obvious, and a large number of old laser spots. After the silicone oil in the eye was removed, the eye examination: the naked visual acuity of the left eye was 0.12, the corrected visual acuity was -3.25ds / - 1.50dc * 180 = 0.4, the intraocular pressure was 19mmHg, the ring ridge was seen in the fundus after mydriasis, a large number of old laser spots and flat retina;The OCT showed that the macular structure of the left eye was complete and there was no subretinal fluid. Conclusion: Scleral cerclage + padding + vitrectomy + silicone oil filling is an effective treatment for retinal detachment (RD) after PIOL.

Stochastic gradient descent algorithm is a classical and useful method for stochastic optimisation. While stochastic gradient descent has been theoretically investigated for decades and successfully applied in machine learning such as training of deep neural networks, it essentially relies on obtaining the unbiased estimates of gradients/subgradients of the objective functions. In this paper, by constructing the randomised differences of the objective function, a gradient‐free algorithm, named the stochastic randomised‐difference descent algorithm, is proposed for stochastic convex optimisation. Under the strongly convex assumption of the objective function, it is proved that the estimates generated from stochastic randomised‐difference descent converge to the optimal value with probability one, and the convergence rates of both the mean square error of estimates and the regret functions are established. Finally, some numerical examples are prsented.

In many real-world scenarios, extrinsic rewards provided by the environment are sparse. An agent trained with classic reinforcement learning algorithm fails to explore these environments in a sufficient and effective way. To address this problem, the exploration bonus which derives from environmental novelty serves as intrinsic motivation for the agent. In recent years, curiosity-driven exploration is a mainstream approach to describe environmental novelty through prediction errors of dynamics models. Due to the expressive ability limitations of curiosity-based environmental novelty and the difficulty of finding appropriate feature space, most curiosity-driven exploration methods have the problem of overprotection against repetition. This problem can reduce the efficiency of exploration and lead the agent into a trap with local optimality. In this article, we propose a combination of persisting curiosity and temporary curiosity framework to deal with the problem of overprotection against repetition. We introduce the self-attention mechanism from the field of computer vision and propose a sequence-based self-attention mechanism for temporary curiosity generation. We compare our framework with some previous exploration methods in hard-exploration environments, provide a series of comprehensive analysis of the proposed framework and investigate the effect of the individual components of our method. The experimental results indicate that the proposed framework delivers superior performance than existing methods.

Objective: To analyze the prevalence of hand, foot and mouth disease (HFMD) in China during 2008-2018 and to provide reference for the prevention and control of HFMD. Methods: Literature eligible for selection was retrieved from the China's national knowledge infrastructure project (CNKI), and data from 2008-2018 Chinese HFMD reports were systematically analyzed. Excel and SPSS 20.00 were used for statistical analysis, and GraphPad Prism 6 was used for mapping. Results: The analysis showed that the average incidence rate of HFMD in China from 2008 to 2018 was 123.37±93.14 cases per 100,000 people, with a male-to-female ratio of (1.50±0.15):1, and the vast majority of cases were concentrated in the under-5 age group, mainly in scattered children. The time of onset of HFMD showed obvious seasonal characteristics, in which single-peak provinces were higher than double-peak provinces, with double-peak trends in April-July and October-November, respectively, and single-peak trends mainly in May-July. Based on the spatial distribution study, the incidence pattern showed obvious regional differences (P

AIM: To investigate the therapeutic effect of 23G vitrectomy system used in children with congenital cataracts.METHODS: From November 2017 to December 2018, 11 children(19 eyes)aged 3-8 years old with congenital cataract were recruited in the Department of Ophthalmology, the First Affiliated Hospital of Hainan Medical University. 23G vitrectomy instruments were used to enter the anterior chamber through the incision at the corneal limbus, and complete ring resection of anterior capsule, cataract aspiration, intraocular lens implantation, and ring resection of posterior capsule combined with anterior vitrectomy. The visual acuity, intraocular pressure and ocular conditions of the children were observed during 3-24mo of follow-up.RESULTS: All the surgeries were uneventful. During the surgeries, the anterior chambers were stable, the ring resections of lens capsules were accurate, and the IOLs were implanted at the 1st phase. The postoperative visual acuity was significantly improved compared with that before surgery. Only 1 patient had intraocular pressure higher than 25mmHg after surgery, which was controlled within the normal range after medication was given. During the follow-up period, all the children had clear corneas, no anterior chamber hemorrhage, no pupil deformation, the intraocular lens was in the center and the right position, the optic axis was transparent, and no obvious complications occurred.CONCLUSION: The 23G vitrectomy system is safe and effective for the surgery of congenital cataract, it can achieve little trauma, accurate lens capsule resection, little postoperative reaction and few complications.

In this study, a rapid and sensitive analytical method has been developed to detect tetracycline hydrochloride (TC) using manganese-doped carbon dots (Mn-CDs) prepared by one-step hydrothermal procedure using 1-(2-pyridinylazo)-2-naohthalenol (PAN) and MnCl2 as precursor reagents. The obtained Mn-CDs showed an ultraviolet emission at 360 nm with an excitation wavelength of 300 nm. TC has a strong characteristic absorption peak at 356 nm, which has a large spectral overlap with the emission band of the Mn-CDs. The fluorescence intensity (FI) of Mn-CDs at 360 nm is linearly quenched within the TC concentration range of 0.1-200 µM. The developed assay for the detection of TC was based on an inner filter effect (IFE) mechanism and is rapid, sensitive, and was successfully applied for the determination of TC in different poultry meat samples with satisfactory results.

Dimension reduction (DR)-based on extreme learning machine auto-encoder (ELM-AE) has achieved many successes in recent years. By minimizing the self-reconstruction error, the ELM-AE-based DR algorithms learn the compressed representations which facilitate the subsequent classification. However, the existing ELM-AEs only consider the DR problem in an unsupervised manner and ignore the valuable supervised information when these information is available. To find discriminative features of the original data, in this paper, we propose a graph embedding-based DR framework with ELM (GDR-ELM) for DR problems. Instead of self-reconstruction, the proposed GDR-ELM reconstructs all samples according to the weights in a graph matrix containing the supervised information. Furthermore, GDR-ELM can be stacked as building blocks to construct a multilayer framework like other ELM-AEs for more complicated representation learning tasks. Experiments on various datasets demonstrate the effectiveness of the proposed GDR-ELM and its multilayer framework.

BACKGROUND Strategies for integrating artificial intelligence (AI) into thyroid nodule management require additional development and testing. We developed a deep-learning AI model (ThyNet) to differentiate between malignant tumours and benign thyroid nodules and aimed to investigate how ThyNet could help radiologists improve diagnostic performance and avoid unnecessary fine needle aspiration. METHODS ThyNet was developed and trained on 18 049 images of 8339 patients (training set) from two hospitals (the First Affiliated Hospital of Sun Yat-sen University, Guangzhou, China, and Sun Yat-sen University Cancer Center, Guangzhou, China) and tested on 4305 images of 2775 patients (total test set) from seven hospitals (the First Affiliated Hospital of Guangzhou University of Chinese Medicine, Guangzhou, China; the Sixth Affiliated Hospital of Sun Yat-sen University, Guangzhou, China; the Guangzhou Army General Hospital, Guangzhou, China; the Third Affiliated Hospital of Sun Yat-sen University, Guangzhou, China; the First Affiliated Hospital of Sun Yat-sen University; Sun Yat-sen University Cancer Center; and the First Affiliated Hospital of Guangxi Medical University, Nanning, China) in three stages. All nodules in the training and total test set were pathologically confirmed. The diagnostic performance of ThyNet was first compared with 12 radiologists (test set A); a ThyNet-assisted strategy, in which ThyNet assisted diagnoses made by radiologists, was developed to improve diagnostic performance of radiologists using images (test set B); the ThyNet assisted strategy was then tested in a real-world clinical setting (using images and videos; test set C). In a simulated scenario, the number of unnecessary fine needle aspirations avoided by ThyNet-assisted strategy was calculated. FINDINGS The area under the receiver operating characteristic curve (AUROC) for accurate diagnosis of ThyNet (0·922 [95% CI 0·910-0·934]) was significantly higher than that of the radiologists (0·839 [0·834-0·844]; p

Despite the popularity of Model Compression and Multitask Learning, how to effectively compress a multitask model has been less thoroughly analyzed due to the challenging entanglement of tasks in the parameter space. In this paper, we propose DiSparse, a simple, effective, and first-of-its-kind multitask pruning and sparse training scheme. We consider each task independently by disentangling the importance measurement and take the unanimous decisions among all tasks when performing parameter pruning and selection. Our experimental results demonstrate superior performance on various configurations and settings compared to popular sparse training and pruning methods. Besides the effectiveness in compression, DiSparse also provides a powerful tool to the multitask learning community. Surprisingly, we even observed better performance than some dedicated multitask learning methods in several cases despite the high model sparsity enforced by DiSparse. We analyzed the pruning masks generated with DiSparse and observed strikingly similar sparse network architecture identified by each task even before the training starts. We also observe the existence of a "watershed" layer where the task relatedness sharply drops, implying no benefits in continued parameters sharing. Our code and models will be available at: https://github.com/SHI-Labs/DiSparse-Multitask-Model-Compression., Accepted at CVPR 2022

Objectives To evaluate the feasibility and advantages of wedge resection plus transverse suture without mesentery detached approach applied to loop ileostomy closure by analyzing the surgical data and the incidence of postoperative complications of patients undergoing this procedure. Methods We performed a retrospective analysis of the hospitalization data of patients who underwent ileostomy closure surgery and met the research standards from January 2017 to April 2021 in Guangxi Medical University Cancer Hospital; all surgeries were performed by the same surgeon. The perioperative data were statistically analyzed by grouping. Results In total, 65 patients were enrolled in this study, with 12 in the wedge resection group, 35 in the stapler group, and 18 in the hand suture group. There was no significant difference in operation time between the wedge resection group and stapler group (P > 0.05), but both groups had shorter operation time than that in the hand suture group (P Conclusions The wedge resection plus transverse suture without mesentery detached approach is safe and easy for closure of loop ileostomy in selected patients, and the intestinal motility recovers rapidly postoperatively. It costs less surgical consumables, and is particularly suitable for the currently implemented Diagnosis-Related Groups payment method.

Efficient and collision-free coordination of two robot arms is increasingly needed in various service-oriented robotic applications. This paper proposes a dual arm coordination algorithm to improve the efficiency of coordination by considering both robot’s actions and operating sequences of the tasks that need to use two arms to complete complex operations. Teleoperation demonstration is first performed to obtain the robot’s human-like motion trajectories, so as to reduce the probability of the collisions between the two arms. The coordination diagram in time domain is then designed to more clearly represent the situations of trajectory collisions and find the collision-free coordination action law. A Coordination Pair Generator (CPG) is designed to reorganize the operating sequences according to the characteristics of input trajectories and the action coordination. The effectiveness and efficiency of our method are verified on the simulation and physical experiments which execute the drug sorting task in nursing homes, respectively, on the ABB YuMi robot model and self-developed robot system. According to the experiment results, the operation time has been reduced by 9% and the collision area has been reduced by 7.5%.

Meta reinforcement learning (meta-RL) is a promising technique for fast task adaptation by leveraging prior knowledge from previous tasks. Recently, context-based meta-RL has been proposed to improve data efficiency by applying a principled framework, dividing the learning procedure into task inference and task execution. However, the task information is not adequately leveraged in this approach, thus leading to inefficient exploration. To address this problem, we propose a novel context-based meta-RL framework with an improved exploration mechanism. For the existing exploration and execution problem in context-based meta-RL, we propose a novel objective that employs two exploration terms to encourage better exploration in action and task embedding space, respectively. The first term pushes for improving the diversity of task inference, while the second term, named action information, works as sharing or hiding task information in different exploration stages. We divide the meta-training procedure into task-independent exploration and task-relevant exploration stages according to the utilization of action information. By decoupling task inference and task execution and proposing the respective optimization objectives in the two exploration stages, we can efficiently learn policy and task inference networks. We compare our algorithm with several popular meta-RL methods on MuJoco benchmarks with both dense and sparse reward settings. The empirical results show that our method significantly outperforms baselines on the benchmarks in terms of sample efficiency and task performance.

Spatial redundancy commonly exists in the learned representations of convolutional neural networks (CNNs), leading to unnecessary computation on high-resolution features. In this paper, we propose a novel Spatially Adaptive feature Refinement (SAR) approach to reduce such superfluous computation. It performs efficient inference by adaptively fusing information from two branches: one conducts standard convolution on input features at a lower spatial resolution , and the other one selectively refines a set of regions at the original resolution. The two branches complement each other in feature learning, and both of them evoke much less computation than standard convolution. SAR is a flexible method that can be conveniently plugged into existing CNNs to establish models with reduced spatial redundancy. Experiments on CIFAR and ImageNet classification, COCO object detection and PASCAL VOC semantic segmentation tasks validate that the proposed SAR can consistently improve the network performance and efficiency. Notably, our results show that SAR only refines less than 40% of the regions in the feature representations of a ResNet for 97% of the samples in the validation set of ImageNet to achieve comparable accuracy with the original model, revealing the high computational redundancy in the spatial dimension of CNNs.

Autophagy has been shown to maintain neural system homeostasis during stroke. However, the molecular mechanisms underlying neuronal autophagy in ischemic stroke remain poorly understood. This study aims to investigate the regulatory mechanisms of the pathway consisting of MEG3 (maternally expressed gene 3), microRNA-378 (miR-378), and GRB2 (growth factor receptor-bound protein 2) in neuronal autophagy and neurological functional impairment in ischemic stroke. A mouse model of the middle cerebral artery occluded–induced ischemic stroke and an in vitro model of oxygen-glucose deprivation–induced neuronal injury were developed. To understand the role of the MEG3/miR-378/GRB2 axis in the neuronal regulation, the expression of proteins associated with autophagy in neurons was measured by Western blotting analysis, and neuron death was evaluated using a lactate dehydrogenase leakage rate test. First, it was found that the GRB2 gene, up-regulated in middle cerebral artery occluded–operated mice and oxygen-glucose deprivation–exposed neurons, was a target gene of miR-378. Next, miR-378 inhibited neuronal loss and neurological functional impairment in mice, as well as neuronal autophagy and neuronal death by silencing of GRB2. Confirmatory in vitro experiments showed that MEG3 could specifically bind to miR-378 and subsequently up-regulate the expression of GRB2, which in turn suppressed the activation of Akt/mTOR pathway. Taken together, these findings suggested that miR-378 might protect against neuronal autophagy and neurological functional impairment and proposed that a MEG3/miR-378/GRB2 regulatory axis contributed to better understanding of the pathophysiology of ischemic stroke.

Aristolochia yachangensis B.G.Huang, Yan Liu & Y.S.Huang, a new species from limestone areas in Guangxi, China, is described and illustrated. It is morphologically most similar to A. fangchi Y.C.Wu ex L.D.Chow & S.M.Hwang, A. petelotii O.C. Schmidt and A. championii Merr. & Chun in shape of leaf blade, anther, gynostemium and inflorescence on old woody stems. However, it can be easily distinguished from the latter by shape of inflorescence, length of upper and lower portions of perianth tube, colour of the limb and throat. A table and a key to distinguish the new species from other morphologically similar Aristolochia species are also provided.

Tremendous research efforts have been made to thrive deep domain adaptation (DA) by seeking domain-invariant features. Most existing deep DA models only focus on aligning feature representations of task-specific layers across domains while integrating a totally shared convolutional architecture for source and target. However, we argue that such strongly-shared convolutional layers might be harmful for domain-specific feature learning when source and target data distribution differs to a large extent. In this paper, we relax a shared-convnets assumption made by previous DA methods and propose a Domain Conditioned Adaptation Network (DCAN), which aims to excite distinct convolutional channels with a domain conditioned channel attention mechanism. As a result, the critical low-level domain-dependent knowledge could be explored appropriately. As far as we know, this is the first work to explore the domain-wise convolutional channel activation for deep DA networks. Moreover, to effectively align high-level feature distributions across two domains, we further deploy domain conditioned feature correction blocks after task-specific layers, which will explicitly correct the domain discrepancy. Extensive experiments on three cross-domain benchmarks demonstrate the proposed approach outperforms existing methods by a large margin, especially on very tough cross-domain learning tasks., Accepted by AAAI 2020

Domain adaptive semantic segmentation attempts to make satisfactory dense predictions on an unlabeled target domain by utilizing the supervised model trained on a labeled source domain. In this work, we propose Semantic-Guided Pixel Contrast (SePiCo), a novel one-stage adaptation framework that highlights the semantic concepts of individual pixels to promote learning of class-discriminative and class-balanced pixel representations across domains, eventually boosting the performance of self-training methods. Specifically, to explore proper semantic concepts, we first investigate a centroid-aware pixel contrast that employs the category centroids of the entire source domain or a single source image to guide the learning of discriminative features. Considering the possible lack of category diversity in semantic concepts, we then blaze a trail of distributional perspective to involve a sufficient quantity of instances, namely distribution-aware pixel contrast, in which we approximate the true distribution of each semantic category from the statistics of labeled source data. Moreover, such an optimization objective can derive a closed-form upper bound by implicitly involving an infinite number of (dis)similar pairs, making it computationally efficient. Extensive experiments show that SePiCo not only helps stabilize training but also yields discriminative representations, making significant progress on both synthetic-to-real and daytime-to-nighttime adaptation scenarios., Accepted by IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI). Code is available at https://github.com/BIT-DA/SePiCo