Search

Lifelong few-shot customization for text-to-image diffusion aims to continually generalize existing models for new tasks with minimal data while preserving old knowledge. Current customization diffusion models excel in few-shot tasks but struggle with catastrophic forgetting problems in lifelong generations. In this study, we identify and categorize the catastrophic forgetting problems into two folds: relevant concepts forgetting and previous concepts forgetting. To address these challenges, we first devise a data-free knowledge distillation strategy to tackle relevant concepts forgetting. Unlike existing methods that rely on additional real data or offline replay of original concept data, our approach enables on-the-fly knowledge distillation to retain the previous concepts while learning new ones, without accessing any previous data. Second, we develop an In-Context Generation (ICGen) paradigm that allows the diffusion model to be conditioned upon the input vision context, which facilitates the few-shot generation and mitigates the issue of previous concepts forgetting. Extensive experiments show that the proposed Lifelong Few-Shot Diffusion (LFS-Diffusion) method can produce high-quality and accurate images while maintaining previously learned knowledge.

The Stellar Abundances and Galactic Evolution Survey (SAGES) is a multi-band survey that covers the northern sky area of ~12000 deg2. Nanshan One-meter Wide-field Telescope (NOWT) of Xinjiang Astronomical Observatory (XAO) carried out observations on g/r/i bands. We present here the survey strategy, data processing, catalog construction, and database schema. The observations of NOWT started in 2016 August and was completed in 2018 January, total 17827 frames were obtained and ~4600 deg2 sky areas were covered. In this paper, we released the catalog of the data in the g/r/i bands observed with NOWT. In total, there are 109,197,578 items of the source records. The catalog is the supplement for the SDSS for the bright end, and the combination of our catalog and these catalogs could be helpful for source selections for other surveys and the Milky Way sciences, e.g., white dwarf candidates and stellar flares., Comment: 12 pages, 8 figures, accepted for publication in RAA

Motion forecasting for agents in autonomous driving is highly challenging due to the numerous possibilities for each agent's next action and their complex interactions in space and time. In real applications, motion forecasting takes place repeatedly and continuously as the self-driving car moves. However, existing forecasting methods typically process each driving scene within a certain range independently, totally ignoring the situational and contextual relationships between successive driving scenes. This significantly simplifies the forecasting task, making the solutions suboptimal and inefficient to use in practice. To address this fundamental limitation, we propose a novel motion forecasting framework for continuous driving, named RealMotion. It comprises two integral streams both at the scene level: (1) The scene context stream progressively accumulates historical scene information until the present moment, capturing temporal interactive relationships among scene elements. (2) The agent trajectory stream optimizes current forecasting by sequentially relaying past predictions. Besides, a data reorganization strategy is introduced to narrow the gap between existing benchmarks and real-world applications, consistent with our network. These approaches enable exploiting more broadly the situational and progressive insights of dynamic motion across space and time. Extensive experiments on Argoverse series with different settings demonstrate that our RealMotion achieves state-of-the-art performance, along with the advantage of efficient real-world inference. The source code will be available at https://github.com/fudan-zvg/RealMotion., Comment: Accepted at NeurIPS 2024 Spotlight

Accurate motion forecasting for traffic agents is crucial for ensuring the safety and efficiency of autonomous driving systems in dynamically changing environments. Mainstream methods adopt a one-query-one-trajectory paradigm, where each query corresponds to a unique trajectory for predicting multi-modal trajectories. While straightforward and effective, the absence of detailed representation of future trajectories may yield suboptimal outcomes, given that the agent states dynamically evolve over time. To address this problem, we introduce DeMo, a framework that decouples multi-modal trajectory queries into two types: mode queries capturing distinct directional intentions and state queries tracking the agent's dynamic states over time. By leveraging this format, we separately optimize the multi-modality and dynamic evolutionary properties of trajectories. Subsequently, the mode and state queries are integrated to obtain a comprehensive and detailed representation of the trajectories. To achieve these operations, we additionally introduce combined Attention and Mamba techniques for global information aggregation and state sequence modeling, leveraging their respective strengths. Extensive experiments on both the Argoverse 2 and nuScenes benchmarks demonstrate that our DeMo achieves state-of-the-art performance in motion forecasting., Comment: NeurIPS 2024

Existing top-performance autonomous driving systems typically rely on the multi-modal fusion strategy for reliable scene understanding. This design is however fundamentally restricted due to overlooking the modality-specific strengths and finally hampering the model performance. To address this limitation, in this work, we introduce a novel modality interaction strategy that allows individual per-modality representations to be learned and maintained throughout, enabling their unique characteristics to be exploited during the whole perception pipeline. To demonstrate the effectiveness of the proposed strategy, we design DeepInteraction++, a multi-modal interaction framework characterized by a multi-modal representational interaction encoder and a multi-modal predictive interaction decoder. Specifically, the encoder is implemented as a dual-stream Transformer with specialized attention operation for information exchange and integration between separate modality-specific representations. Our multi-modal representational learning incorporates both object-centric, precise sampling-based feature alignment and global dense information spreading, essential for the more challenging planning task. The decoder is designed to iteratively refine the predictions by alternately aggregating information from separate representations in a unified modality-agnostic manner, realizing multi-modal predictive interaction. Extensive experiments demonstrate the superior performance of the proposed framework on both 3D object detection and end-to-end autonomous driving tasks. Our code is available at https://github.com/fudan-zvg/DeepInteraction., Comment: Journal extension of NeurIPS 2022. arXiv admin note: text overlap with arXiv:2208.11112

We present the discoveries of two giant planets orbiting the red giant branch (RGB) star HD 112570 and the red clump (RC) star HD 154391, based on the radial velocity (RV) measurements from Xinglong station and Okayama Astrophysical Observatory (OAO). Spectroscopic and asteroseismic analyses suggest that HD 112570 has a mass of $1.15\pm0.12\,M_{\odot}$, a radius of $9.85\pm0.23\,R_{\odot}$, a metallicity [Fe/H] of $-0.46\pm0.1$ and a ${\rm log}\,g$ of $2.47\pm0.1$. With the joint analysis of RV and Hipparcos-Gaia astrometry, we obtain a dynamical mass of $M_{\rm p}={3.42}_{-0.84}^{+1.4}\ M_{\rm Jup}$, a period of $P={2615}_{-77}^{+85}$ days and a moderate eccentricity of $e={0.20}_{-0.14}^{+0.16}$ for the Jovian planet HD 112570 b. For HD 154391, it has a mass of $2.07\pm0.03\,M_{\odot}$, a radius of $8.56\pm0.05\,R_{\odot}$, a metallicity [Fe/H] of $0.07\pm0.1$ and a ${\rm log}\,g$ of $2.86\pm0.1$. The super-Jupiter HD 154391 b has a mass of $M_{\rm p}={9.1}_{-1.9}^{+2.8}\ M_{\rm Jup}$, a period of $P={5163}_{-57}^{+60}$ days and an eccentricity of $e={0.20}_{-0.04}^{+0.04}$. We found HD 154391 b has one of the longest orbital period among those ever discovered orbiting evolved stars, which may provide a valuable case in our understanding of planetary formation at wider orbits. Moreover, while a mass gap at $4\,M_{\rm Jup}$ seems to be present in the population of giant stars, there appears to be no significant differences in the distribution of metallicity among giant planets with masses above or below this threshold. Finally, The origin of the abnormal accumulation near 2 au for planets around large evolved stars ($R_{\star}>21\,R_{\odot}$), remains unclear., Comment: 26 pages, 18 figures, 8 tables, Accepted for publication in The Astronomical Journal

Resting-state fMRI (rs-fMRI) functional connectivity (FC) analysis provides valuable insights into the relationships between different brain regions and their potential implications for neurological or psychiatric disorders. However, specific design efforts to predict treatment response from rs-fMRI remain limited due to difficulties in understanding the current brain state and the underlying mechanisms driving the observed patterns, which limited the clinical application of rs-fMRI. To overcome that, we propose a graph learning framework that captures comprehensive features by integrating both correlation and distance-based similarity measures under a contrastive loss. This approach results in a more expressive framework that captures brain dynamic features at different scales and enables more accurate prediction of treatment response. Our experiments on the chronic pain and depersonalization disorder datasets demonstrate that our proposed method outperforms current methods in different scenarios. To the best of our knowledge, we are the first to explore the integration of distance-based and correlation-based neural similarity into graph learning for treatment response prediction., Comment: Proceedings of the 2023 International Conference on Neural Information Processing (ICONIP)

Aspect-based sentiment analysis is a long-standing research interest in the field of opinion mining, and in recent years, researchers have gradually shifted their focus from simple ABSA subtasks to end-to-end multi-element ABSA tasks. However, the datasets currently used in the research are limited to individual elements of specific tasks, usually focusing on in-domain settings, ignoring implicit aspects and opinions, and with a small data scale. To address these issues, we propose a large-scale Multi-Element Multi-Domain dataset (MEMD) that covers the four elements across five domains, including nearly 20,000 review sentences and 30,000 quadruples annotated with explicit and implicit aspects and opinions for ABSA research. Meanwhile, we evaluate generative and non-generative baselines on multiple ABSA subtasks under the open domain setting, and the results show that open domain ABSA as well as mining implicit aspects and opinions remain ongoing challenges to be addressed. The datasets are publicly released at \url{https://github.com/NUSTM/MEMD-ABSA}.

The Stellar Abundances and Galactic Evolution Survey (SAGES) of the northern sky is a specifically-designed multi-band photometric survey aiming to provide reliable stellar parameters with accuracy comparable to those from low-resolution optical spectra. It was carried out with the 2.3-m Bok telescope of Steward Observatory and three other telescopes. The observations in the $u_s$ and $v_s$ passband produced over 36,092 frames of images in total, covering a sky area of $\sim9960$ degree$^2$. The median survey completeness of all observing fields for the two bands are of $u_{\rm s}=20.4$ mag and $v_s=20.3$ mag, respectively, while the limiting magnitudes with signal-to-noise ratio (S/N) of 100 are $u_s\sim17$ mag and $v_s\sim18$ mag, correspondingly. We combined our catalog with the data release 1 (DR1) of the first of Panoramic Survey Telescope And Rapid Response System (Pan-STARRS1, PS1) catalog, and obtained a total of 48,553,987 sources which have at least one photometric measurement in each of the SAGES $u_s$ and $v_s$ and PS1 $grizy$ passbands, which is the DR1 of SAGES and it will be released in our paper. We compare our $gri$ point-source photometry with those of PS1 and found an RMS scatter of $\sim2$% in difference of PS1 and SAGES for the same band. We estimated an internal photometric precision of SAGES to be on the order of $\sim1$%. Astrometric precision is better than $0^{\prime\prime}.2$ based on comparison with the DR1 of Gaia mission. In this paper, we also describe the final end-user database, and provide some science applications., Comment: 49 pages, 21 figures, 5 table, accepted for publication in ApJS

Few-shot open-set recognition aims to classify both seen and novel images given only limited training data of seen classes. The challenge of this task is that the model is required not only to learn a discriminative classifier to classify the pre-defined classes with few training data but also to reject inputs from unseen classes that never appear at training time. In this paper, we propose to solve the problem from two novel aspects. First, instead of learning the decision boundaries between seen classes, as is done in standard close-set classification, we reserve space for unseen classes, such that images located in these areas are recognized as the unseen classes. Second, to effectively learn such decision boundaries, we propose to utilize the background features from seen classes. As these background regions do not significantly contribute to the decision of close-set classification, it is natural to use them as the pseudo unseen classes for classifier learning. Our extensive experiments show that our proposed method not only outperforms multiple baselines but also sets new state-of-the-art results on three popular benchmarks, namely tieredImageNet, miniImageNet, and Caltech-USCD Birds-200-2011 (CUB)., Comment: Accpeted to ACM MM 2022

Few-shot class-incremental learning (FSCIL) aims to design machine learning algorithms that can continually learn new concepts from a few data points, without forgetting knowledge of old classes. The difficulty lies in that limited data from new classes not only lead to significant overfitting issues but also exacerbate the notorious catastrophic forgetting problems. Moreover, as training data come in sequence in FSCIL, the learned classifier can only provide discriminative information in individual sessions, while FSCIL requires all classes to be involved for evaluation. In this paper, we address the FSCIL problem from two aspects. First, we adopt a simple but effective decoupled learning strategy of representations and classifiers that only the classifiers are updated in each incremental session, which avoids knowledge forgetting in the representations. By doing so, we demonstrate that a pre-trained backbone plus a non-parametric class mean classifier can beat state-of-the-art methods. Second, to make the classifiers learned on individual sessions applicable to all classes, we propose a Continually Evolved Classifier (CEC) that employs a graph model to propagate context information between classifiers for adaptation. To enable the learning of CEC, we design a pseudo incremental learning paradigm that episodically constructs a pseudo incremental learning task to optimize the graph parameters by sampling data from the base dataset. Experiments on three popular benchmark datasets, including CIFAR100, miniImageNet, and Caltech-USCD Birds-200-2011 (CUB200), show that our method significantly outperforms the baselines and sets new state-of-the-art results with remarkable advantages., Comment: Accpeted to CVPR2021

Star clusters are good tracers for formation and evolution of galaxies. We compared different fitting methods by using spectra (or by combining photometry) to determine the physical parameters. We choose a sample of 17 star clusters in M33, which previously lacked spectroscopic observations. The low-resolution spectra were taken with the Xinglong 2.16-m reflector of NAOC. The photometry used in the fitting includes $\rm u_{SC}$ and $\rm v_{SAGE}$ bands from the SAGE survey, as well as the published $UBVRI$ and $ugriz$ photometry. We firstly derived ages and metallicities with the {\sc ULySS} (Vazdekis et al. and {\sc pegase-hr}) SSP model and the Bruzual \& Charlot (2003) (BC03) stellar population synthesis models for the full-spectrum fitting. The fitting results of both the BC03 and {\sc ULySS} models seem consistent with those of previous works as well. Then we add the SAGE $\rm u_{SC}$ and $\rm v_{SAGE}$ photometry in the spectroscopic fitting with the BC03 models. It seems the results become much better, especially for the Padova 2000+Chabrier IMF set. Finally we add more photometry data, $UBVRI$ and $ugriz$, in the fitting and we found that the results do not improve significantly. Therefore, we conclude that the photometry is useful for improving the fitting results, especially for the blue bands ($\lambda <4000$ {\AA}), e.g., $\rm u_{SC}$ and $\rm v_{SAGE}$ band. At last, we discuss the "UV-excess" for the star clusters and we find five star clusters have UV-excess, based on the $GALEX$ FUV, NUV photometry., Comment: Accepted for publication in ApJS

In heterogeneous networks (HetNets), how to improve spectrum efficiency is a crucial issue. Meanwhile increased energy consumption inspires network operators to deploy renewable energy sources as assistance to traditional electricity. Based on above aspects, we allow base stations (BSs) to share their licensed spectrum resource with each other and adjust transmission power to adapt to the renewable energy level. Considering the sharing fairness among BSs, we formulate a multi-person bargaining problem as a stochastic optimization problem. We divide the optimization problem into three parts: data rate control, resource allocation and energy management. An online dynamic control algorithm is proposed to control admission rate and resource allocation to maximize the transmission and sharing profits with the least grid energy consumption. Simulation results investigate the time-varying data control and energy management of BSs and demonstrate the effectiveness of the proposed scheme., Comment: 29 pages, 7 figures