Search

Text-to-image generative models can produce diverse high-quality images of concepts with a text prompt, which have demonstrated excellent ability in image generation, image translation, etc. We in this work study the problem of synthesizing instantiations of a user's own concepts in a never-ending manner, i.e., create your world, where the new concepts from user are quickly learned with a few examples. To achieve this goal, we propose a Lifelong text-to-image Diffusion Model (L $^{2}$2 DM), which intends to overcome knowledge “catastrophic forgetting” for the past encountered concepts, and semantic “catastrophic neglecting” for one or more concepts in the text prompt. In respect of knowledge “catastrophic forgetting”, our L $^{2}$2 DM framework devises a task-aware memory enhancement module and an elastic-concept distillation module, which could respectively safeguard the knowledge of both prior concepts and each past personalized concept. When generating images with a user text prompt, the solution to semantic “catastrophic neglecting” is that a concept attention artist module can alleviate the semantic neglecting from concept aspect, and an orthogonal attention module can reduce the semantic binding from attribute aspect. To the end, our model can generate more faithful image across a range of continual text prompts in terms of both qualitative and quantitative metrics, when comparing with the related state-of-the-art models.

The visual perception systems aim to autonomously collect consecutive visual data and perceive the relevant information online like human beings. In comparison with the classical static visual systems focusing on fixed tasks (e.g., face recognition for visual surveillance), the real-world visual systems (e.g., the robot visual system) often need to handle unpredicted tasks and dynamically changed environments, which need to imitate human-like intelligence with open-ended online learning ability. Therefore, we provide a comprehensive analysis of open-ended online learning problems for autonomous visual perception in this survey. Based on “what to online learn” among visual perception scenarios, we classify the open-ended online learning methods into five categories: instance incremental learning to handle data attributes changing, feature evolution learning for incremental and decremental features with the feature dimension changed dynamically, class incremental learning and task incremental learning aiming at online adding new coming classes/tasks, and parallel and distributed learning for large-scale data to reveal the computational and storage advantages. We discuss the characteristic of each method and introduce several representative works as well. Finally, we introduce some representative visual perception applications to show the enhanced performance when using various open-ended online learning models, followed by a discussion of several future directions.

Continual learning algorithms which keep the parameters of new tasks close to that of previous tasks, are popular in preventing catastrophic forgetting in sequential task learning settings. However, 1) the performance for the new continual learner will be degraded without distinguishing the contributions of previously learned tasks; 2) the computational cost will be greatly increased with the number of tasks, since most existing algorithms need to regularize all previous tasks when learning new tasks. To address the above challenges, we propose a self-paced Weight Consolidation (spWC) framework to attain robust continual learning via evaluating the discriminative contributions of previous tasks. To be specific, we develop a self-paced regularization to reflect the priorities of past tasks via measuring difficulty based on key performance indicator (i.e., accuracy). When encountering a new task, all previous tasks are sorted from “difficult” to “easy” based on the priorities. Then the parameters of the new continual learner will be learned via selectively maintaining the knowledge amongst more difficult past tasks, which could well overcome catastrophic forgetting with less computational cost. We adopt an alternative convex search to iteratively update the model parameters and priority weights in the bi-convex formulation. The proposed spWC framework is plug-and-play, which is applicable to most continual learning algorithms (e.g., EWC, MAS and RCIL) in different directions (e.g., classification and segmentation). Experimental results on several public benchmark datasets demonstrate that our proposed framework can effectively improve performance when compared with other popular continual learning algorithms.

Federated learning (FL) is a hot collaborative training framework via aggregating model parameters of decentralized local clients. However, most FL methods unreasonably assume data categories of FL framework are known and fixed in advance. Moreover, some new local clients that collect novel categories unseen by other clients may be introduced to FL training irregularly. These issues render global model to undergo catastrophic forgetting on old categories, when local clients receive new categories consecutively under limited memory of storing old categories. To tackle the above issues, we propose a novel Local-Global Anti-forgetting (LGA) model. It ensures no local clients are left behind as they learn new classes continually, by addressing local and global catastrophic forgetting. Specifically, considering tackling class imbalance of local client to surmount local forgetting, we develop a category-balanced gradient-adaptive compensation loss and a category gradient-induced semantic distillation loss. They can balance heterogeneous forgetting speeds of hard-to-forget and easy-to-forget old categories, while ensure consistent class-relations within different tasks. Moreover, a proxy server is designed to tackle global forgetting caused by Non-IID class imbalance between different clients. It augments perturbed prototype images of new categories collected from local clients via self-supervised prototype augmentation, thus improving robustness to choose the best old global model for local-side semantic distillation loss. Experiments on representative datasets verify superior performance of our model against comparison methods.

Unsupervised domain adaptation without accessing expensive annotation processes of target data has achieved remarkable successes in semantic segmentation. However, most existing state-of-the-art methods cannot explore whether semantic representations across domains are transferable or not, which may result in the negative transfer brought by irrelevant knowledge. To tackle this challenge, in this paper, we develop a novel Knowledge Aggregation-induced Transferability Perception (KATP) module for unsupervised domain adaptation, which is a pioneering attempt to distinguish transferable or untransferable knowledge across domains. Specifically, the KATP module is designed to quantify which semantic knowledge across domains is transferable, by incorporating the transferability information propagation from constructed global category-wise prototypes. Based on KATP, we design a novel KATP Adaptation Network (KATPAN) to determine where and how to transfer. The KATPAN contains a transferable appearance translation module $\mathcal {T}_A(\cdot)$TA(·) and a transferable representation augmentation module $\mathcal {T}_R(\cdot)$TR(·), where both modules construct a virtuous circle of performance promotion. $\mathcal {T}_A(\cdot)$TA(·) develops a transferability-aware information bottleneck to highlight where to adapt transferable visual characterizations and modality information; $\mathcal {T}_R(\cdot)$TR(·) explores how to augment transferable representations while abandoning untransferable information, and promotes the translation performance of $\mathcal {T}_A(\cdot)$TA(·) in return. Comprehensive experiments on several representative benchmark datasets and a medical dataset support the state-of-the-art performance of our model.

3-D object recognition has successfully become an appealing research topic in the real world. However, most existing recognition models unreasonably assume that the categories of 3-D objects cannot change over time in the real world. This unrealistic assumption may result in significant performance degradation for them to learn new classes of 3-D objects consecutively due to the catastrophic forgetting on old learned classes. Moreover, they cannot explore which 3-D geometric characteristics are essential to alleviate the catastrophic forgetting on old classes of 3-D objects. To tackle the above challenges, we develop a novel Incremental 3-D Object Recognition Network (i.e., InOR-Net), which could recognize new classes of 3-D objects continuously by overcoming the catastrophic forgetting on old classes. Specifically, category-guided geometric reasoning is proposed to reason local geometric structures with distinctive 3-D characteristics of each class by leveraging intrinsic category information. We then propose a novel critic-induced geometric attention mechanism to distinguish which 3-D geometric characteristics within each class are beneficial to overcome the catastrophic forgetting on old classes of 3-D objects while preventing the negative influence of useless 3-D characteristics. In addition, a dual adaptive fairness compensations’ strategy is designed to overcome the forgetting brought by class imbalance by compensating biased weights and predictions of the classifier. Comparison experiments verify the state-of-the-art performance of the proposed InOR-Net model on several public point cloud datasets.

This work presents a novel visual-tactile fused clustering framework, called Lifelong Visual-Tactile Spectral Clustering (i.e., LVTSC), to effectively learn consecutive object clustering tasks for robotic perception. Lifelong learning has become an important and hot topic in recent studies on machine learning, aiming to imitate “human learning” and reduce the computational cost when consecutively learning new tasks. Our proposed LVTSC model explores the knowledge transfer and representation correlation from a local modality-invariant perspective under modality-consistent constraint guidance. For the modality-invariant part, we design a set of modality-invariant basis libraries to capture the latent clustering centers of each modality and a set of modality-invariant feature libraries to forcibly embed the manifold information of each modality. A modal-consistent constraint reinforces the correlation between visual and tactile modalities by maximizing the feature manifold correspondences. When the object clustering task comes continuously, the overall objective is optimized by an effective alternating direction method with guaranteed convergence. Our proposed LVTSC framework has been extensively validated for its effectiveness and efficiency on the three challenging real-world robotic object perception datasets.

Traditional 3D point cloud classification tasks focus on training a classifier in the closed-set scenario, where training and test data have the same label set and the same data distribution. In this work, we focus on a more challenging and realistic scenario in 3D point cloud classification task: universal domain adaptation (UniDA), where 1) data distributions for training and test data are different; and 2) for given label sets of training data and test data, they may have the shared classes and keep the private classes respectively, introducing an extra label set discrepancy. To solve UniDA problem, researchers have designed many methods based on 2D image datasets. However, due to the difficulty in capturing discriminative local geometric structures brought by the unordered and irregular 3D point cloud data, we cannot directly deploy the existing methods based on 2D image datasets to the 3D scenarios. To address UniDA in 3D scenarios, we develop a 3D universal domain adaptation framework, which consists of three modules: Self-Constructed Geometric (SCG) module, Local-to-Global Hypersphere Reasoning (LGHR) module and Self-Supervised Boundary Adaptation (SBA) module. SCG and LGHR generate the discriminative representation, which is used to acquire domain-invariant knowledge for training and test data. SBA is designed to automatically recognize whether a given label is from the shared label set or private label set, and adapts training and test data from the shared label set. To our best knowledge, this work is the first exploration of UniDA for 3D scenarios. Extensive experiments on public 3D point cloud datasets verify that the proposed method outperforms the existing UniDA methods.

Fibrosis assessment performed using 5-T myocardial late gadolinium enhancement MRI had higher signal-to-noise and contrast-to-noise ratios compared with assessment performed at 3-T MRI, with no evidence of differences in subjective image quality and quantitative assessment.

Graphical abstract:

Consider the lifelong machine learning paradigm whose objective is to learn a sequence of tasks depending on previous experiences, e.g., knowledge library or deep network weights. However, the knowledge libraries or deep networks for most recent lifelong learning models are of prescribed size and can degenerate the performance for both learned tasks and coming ones when facing with a new task environment (cluster). To address this challenge, we propose a novel incremental clustered lifelong learning framework with two knowledge libraries: feature learning library and model knowledge library, called Flexible Clustered Lifelong Learning (FCL3). Specifically, the feature learning library modeled by an autoencoder architecture maintains a set of representation common across all the observed tasks, and the model knowledge library can be self-selected by identifying and adding new representative models (clusters). When a new task arrives, our FCL3 model firstly transfers knowledge from these libraries to encode the new task, i.e., effectively and selectively soft-assigning this new task to multiple representative models over feature learning library. Then: 1) the new task with a higher outlier probability will be judged as a new representative, and used to redefine both feature learning library and representative models over time; or 2) the new task with lower outlier probability will only refine the feature learning library. For model optimization, we cast this lifelong learning problem as an alternating direction minimization problem as a new task comes. Finally, we evaluate the proposed framework by analyzing several multitask data sets, and the experimental results demonstrate that our FCL3 model can achieve better performance than most lifelong learning frameworks, even batch clustered multitask learning models. [ABSTRACT FROM AUTHOR]

• NiCo 2 O 4 /rGO nanocomposites with hierarchical and laminated structure was fabricated by one-step ultrasonic spray. • In-situ grown NiCo 2 O 4 nanoparticles act as spacers to prevent re-stacking of graphene. • Tight adhesion between active materials and Ni foam guarantees efficient electron transfer and alleviate peeling of active materials. • NiCo 2 O 4 /rGO composite exhibits a high specific capacitance and excellent cycling stability. • The assembled asymmetric supercapacitor manifests an ultralong cycling lifetime up to 30,000 cycles. Durable and cost-effective electrode materials are essential for practical application of supercapacitors. Herein, large area NiCo 2 O 4 /reduced graphene oxide (NiCo 2 O 4 /rGO) composites with hierarchical structure were fabricated by a facile one-step ultrasonic spray on Ni foam and directly used as the binder-free electrodes for supercapacitors in aqueous KOH electrolyte. Owing to high electrical conductivity of rGO, hierarchical and layered structure of the electrode, as well as tight adhesion of active materials on the current collector, the as-obtained hybrid electrodes show a high specific capacitance of 871 F g−1 at current density of 1 A g−1, good rate performance and remarkable cycling stability with a capacitance retention of 134% after 30000 cycles. Besides, the assembled NiCo 2 O 4 /rGO//AC asymmetric supercapacitor (ASC) displays the maximum energy density of 29.3 Wh kg−1 at a power density of 790.8 W kg−1. Significantly, an ultralong cycling life of 102% capacitance retention is achieved for the ASC device after 30,000 charge/discharge cycles at 20 A g−1. The scalable fabrication route and excellent electrochemical performance of the NiCo 2 O 4 /rGO composites open the door for making novel hybrid electrodes of advanced supercapacitors. [Display omitted] [ABSTRACT FROM AUTHOR]

The state-of-the-art multitask multiview (MTMV) learning tackles a scenario where multiple tasks are related to each other via multiple shared feature views. However, in many real-world scenarios where a sequence of the multiview task comes, the higher storage requirement and computational cost of retraining previous tasks with MTMV models have presented a formidable challenge for this lifelong learning scenario. To address this challenge, in this article, we propose a new continual multiview task learning model that integrates deep matrix factorization and sparse subspace learning in a unified framework, which is termed deep continual multiview task learning (DCMvTL). More specifically, as a new multiview task arrives, DCMvTL first adopts a deep matrix factorization technique to capture hidden and hierarchical representations for this new coming multiview task while accumulating the fresh multiview knowledge in a layerwise manner. Then, a sparse subspace learning model is employed for the extracted factors at each layer and further reveals cross-view correlations via a self-expressive constraint. For model optimization, we derive a general multiview learning formulation when a new multiview task comes and apply an alternating minimization strategy to achieve lifelong learning. Extensive experiments on benchmark data sets demonstrate the effectiveness of our proposed DCMvTL model compared with the existing state-of-the-art MTMV and lifelong multiview task learning models. [ABSTRACT FROM AUTHOR]

To solve the problem of lack of water in arid areas and barren soil in the process of ecological restoration, the growth and drought resistance of Sainfoin (Onobrychis viciifoliaScop.) was studied by inoculation of AMF (arbuscular mycorrhizal fungi), under drought stress. The results show that the inoculation of AMF can significantly improve the growth and mycorrhizal infection rate of Sainfoin and accelerate the absorption of water and nutrients by the roots compared with uninoculated plants. On the 40th day under drought stress, the relative water content, total nitrogen and total phosphorus content of the leaves of inoculated plants increased by 7.27%, 4.21% and 2.40%, respectively, the osmotic adjustment property in plant cells was improved, the protective enzyme system was improved and membrane lipid peroxidation was reduced. The inoculation of AMF can slow down the damage from drought, improve the drought resistance of Sainfoin, and will be of great significance to the ecologic restoration in arid areas.

Human embryonic stem cells (hESCs) are considered to be able to stably maintain their characteristics in vitro for prolonged periods, but we had previously encountered changes in proliferative ability and differentiation potential during extended culture of hESCs. Therefore, we investigated the proliferative ability and differentiation potential of hESCs during long-term culture. The hESCs, SNUhES3, were used to analyze population-doubling time, proliferation rate and differentiation potential. We classified hESCs into three groups according to culture period. Ten colonies of hESCs for each group were daily measured colony area and population-doubling time was assessed by the changes of colony area. Proliferation rate of hESCs was measured by 5-bromo-2'-deoxyuridine (BrdU) assay and telomerase activity. To evaluate differentiation potentials for hESCs, expression levels of undifferentiated and/or differentiated hESCs markers were examined by FACS, RT-PCR and immunostaining. Population-doubling time of early passage hESCs was longer than those of middle or late passage. Proliferative ability of hESCs was accelerated depending on culture periods. Cellular morphologies and the expression level of each three germ layer markers were obviously different from each passage of reattached embryoid bodies (EBs) after spontaneous differentiation. Differentiated cells of late passage expressed higher levels of undifferentiated markers such as Oct4 and SSEA4 than those of early and middle passage. But differentiated cells of early and middle passage expressed higher level of differentiated state markers, Nestin (ectoderm), Brachyury (mesoderm), HNF3ß (endoderm). From these results, it can be inferred that hESCs show higher proliferative abilities and reduced differentiation potentials as the passage number increased. Therefore, we conclude that early passage hESCs could be more suitable than middle and late passage hESCs in differentiation studies.

ABSTRACTSeventy-three analogs of SJ-3366 (1-(3-cyclopenten-1-ylmethyl)-5-ethyl-6-(3,5-dimethylbenzoyl)-2,4(1H,3H)-pyrimidinedione) were synthesized and comparatively evaluated for their ability to inhibit the replication of human immunodeficiency virus type 1 (HIV-1) and HIV-2 and for their ability to suppress virus entry and reverse transcription. These studies were performed to identify inhibitors with activity greater than that of the current lead molecule (SJ-3366) and to utilize structure-activity relationships (SAR) to define the chemical features of the pyrimidinedione congeners responsible for their efficacy, toxicity, and dual mechanism of action against HIV. The results of our SAR evaluations have demonstrated that the addition of the homocyclic moiety at the N-1 of the pyrimidinedione results in acquisition of the ability to inhibit virus entry and extends the range of action of the compounds to include HIV-2. In addition, the results demonstrate that analogs with a methyl linker between the homocyclic substitution and the N-1 of the pyrimidinedione had a greater number of highly active molecules than those analogs possessing ethyl linkers. Six molecules were identified with activity equivalent to or greater than that of SJ-3366, and five additional molecules with highly potent inhibition of reverse transcriptase and virus entry and possessing high efficacy against both HIV-1 and HIV-2 were identified. Six molecules exhibited significant inhibition of viruses with the highly problematic nonnucleoside reverse transcriptase inhibitor (NNRTI) resistance engendering amino acid change K103N in the reverse transcriptase. These evaluations indicate that a new class of NNRTIs has been identified and that these NNRTIs possess highly potent inhibition of HIV-1 with an extended range of action, which now includes HIV-2.

Since the discovery of the 2,4 (1H,3H)-pyrimidinediones as potent non-nucleoside inhibitors of the HIV-1 reverse transcriptase (RT) this class of compounds has yielded a number of N-1 acyclic substituted pyrimidinediones with substantial antiviral activity, which is highly dependent upon their molecular fit into the binding pocket common to this inhibitory class. We have specifically examined the structure activity relationships of compounds with chemical modification made by substituting homocyclic rather than acyclic moieties at N-1 of the pyrimidinedione. Seventy-four compounds were synthesized and evaluated for antiviral activity against HIV-1 and HIV-2. The homocyclic modifications resulted in compounds with significant activity against both HIV-1 and HIV-2, suggesting these compounds represent a new class of non-nucleoside RT inhibitors. The structure-activity relationship (SAR) evaluations indicated that cyclopropyl, phenyl and 1- or 3-cyclopenten-1-yl substitutions at the N-1 of the pyrimidinedione, the addition of a methyl linker between the cyclic moiety and the N-1 and the addition of a benzoyl group at the C-6 of the pyrimidinedione had the greatest contribution to antiviral activity. Five pyrimidinedione analogues with therapeutic indexes (TIs)>450,000 and a specific analogue (1-cyclopropylmethyl-5-isopropyl-6-(3,5-dimethylbenzoyl)-2,4(1H,3H)-pyrimidinedione), which exhibited a TI of >2,000,000, were identified. None of the analogues were cytotoxic to target cells at the highest in vitrotest concentration, which is the upper limit of compound solubility of the analogues in aqueous solution. Thus, we have identified a series of pyrimidinediones with substantially improved antiviral efficacy and range of action and with significantly reduced cellular cytotoxicity.