Search

Current video summarization methods rely heavily on supervised computer vision techniques, which demands time-consuming and subjective manual annotations. To overcome these limitations, we investigated self-supervised video summarization. Inspired by the success of Large Language Models (LLMs), we explored the feasibility in transforming the video summarization task into a Natural Language Processing (NLP) task. By leveraging the advantages of LLMs in context understanding, we aim to enhance the effectiveness of self-supervised video summarization. Our method begins by generating captions for individual video frames, which are then synthesized into text summaries by LLMs. Subsequently, we measure semantic distance between the captions and the text summary. Notably, we propose a novel loss function to optimize our model according to the diversity of the video. Finally, the summarized video can be generated by selecting the frames with captions similar to the text summary. Our method achieves state-of-the-art performance on the SumMe dataset in rank correlation coefficients. In addition, our method has a novel feature of being able to achieve personalized summarization.

In this paper we construct explicit examples that show the sublevel sets of the solution of a $k$-Hessian equation defined on a convex ring do not have to be convex.

It has been more than 30 years since the enigmatic 21 {\mu}m emission feature was first discovered in protoplanetary nebulae (PPNs). Although dozens of different dust carrier candidates have been proposed, there is as yet no widely accepted one. We present the results of molecular observations toward 21{\mu}m objects using the 10m Submillimeter Telescope of Arizona Radio Observatory at the 1.3 mm band and the 13.7 m telescope of Purple Mountain Observatory at the 3mm band, aiming to investigate whether the gas-phase environments of these unusual sources have some peculiarities compared to normal PPNs. We detect 31 emission lines belonging to seven different molecular species, most of which are the first detection in 21 {\mu}m PPNs. The observations provide clues on the identification of the 21 {\mu}m feature. We report a correlation study between the fractional abundance of gas-phase molecules and the strengths of the 21 {\mu}m emission. Our study shows that given the small sample size, the 21 {\mu}m feature has weak or no correlations with the gas-phase molecules. Future radio observations of high spatial and spectral resolution toward a large sample are desirable to elucidate the 21 {\mu}m emission phenomena., Comment: 40 pages, 10 figures, 11 tables, accepted for publication in The Astronomical Journal

Online Continual Learning (CL) solves the problem of learning the ever-emerging new classification tasks from a continuous data stream. Unlike its offline counterpart, in online CL, the training data can only be seen once. Most existing online CL research regards catastrophic forgetting (i.e., model stability) as almost the only challenge. In this paper, we argue that the model's capability to acquire new knowledge (i.e., model plasticity) is another challenge in online CL. While replay-based strategies have been shown to be effective in alleviating catastrophic forgetting, there is a notable gap in research attention toward improving model plasticity. To this end, we propose Collaborative Continual Learning (CCL), a collaborative learning based strategy to improve the model's capability in acquiring new concepts. Additionally, we introduce Distillation Chain (DC), a collaborative learning scheme to boost the training of the models. We adapt CCL-DC to existing representative online CL works. Extensive experiments demonstrate that even if the learners are well-trained with state-of-the-art online CL methods, our strategy can still improve model plasticity dramatically, and thereby improve the overall performance by a large margin. The source code of our work is available at https://github.com/maorong-wang/CCL-DC., Comment: Update Camera-ready revision for CVPR'24

Rydberg microwave (MW) sensors are superior to conventional antenna-based techniques because of their wide operating frequency range and outstanding potential sensitivity. Here, we demonstrate a Rydberg microwave receiver with a high sensitivity of $62\,\mathrm{nV} \mathrm{cm}^{-1} \mathrm{Hz}^{-1/2}$ and broad instantaneous bandwidth of up to $10.2\,\mathrm{MHz}$. Such excellent performance was achieved by the amplification of one generated sideband wave induced by the strong coupling field in the six-wave mixing process of the Rydberg superheterodyne receiver, which was well predicted by our theory. Our system, which possesses a uniquely enhanced instantaneous bandwidth and high-sensitivity features that can be improved further, will promote the application of Rydberg microwave electrometry in radar and communication.

Online Continual Learning (OCL) addresses the problem of training neural networks on a continuous data stream where multiple classification tasks emerge in sequence. In contrast to offline Continual Learning, data can be seen only once in OCL, which is a very severe constraint. In this context, replay-based strategies have achieved impressive results and most state-of-the-art approaches heavily depend on them. While Knowledge Distillation (KD) has been extensively used in offline Continual Learning, it remains under-exploited in OCL, despite its high potential. In this paper, we analyze the challenges in applying KD to OCL and give empirical justifications. We introduce a direct yet effective methodology for applying Momentum Knowledge Distillation (MKD) to many flagship OCL methods and demonstrate its capabilities to enhance existing approaches. In addition to improving existing state-of-the-art accuracy by more than $10\%$ points on ImageNet100, we shed light on MKD internal mechanics and impacts during training in OCL. We argue that similar to replay, MKD should be considered a central component of OCL. The code is available at \url{https://github.com/Nicolas1203/mkd_ocl}., Comment: Accepted to ICML 2024

Open-set semi-supervised object detection (OSSOD) task leverages practical open-set unlabeled datasets that comprise both in-distribution (ID) and out-of-distribution (OOD) instances for conducting semi-supervised object detection (SSOD). The main challenge in OSSOD is distinguishing and filtering the OOD instances (i.e., outliers) during pseudo-labeling since OODs will affect the performance. The only OSSOD work employs an additional offline OOD detection network trained solely with labeled data to solve this problem. However, the limited labeled data restricts the potential for improvement. Meanwhile, the offline strategy results in low efficiency. To alleviate these issues, this paper proposes an end-to-end online OSSOD framework that improves performance and efficiency: 1) We propose a semi-supervised outlier filtering method that more effectively filters the OOD instances using both labeled and unlabeled data. 2) We propose a threshold-free Dual Competing OOD head that further improves the performance by suppressing the error accumulation during semi-supervised outlier filtering. 3) Our proposed method is an online end-to-end trainable OSSOD framework. Experimental results show that our method achieves state-of-the-art performance on several OSSOD benchmarks compared to existing methods. Moreover, additional experiments show that our method is more efficient and can be easily applied to different SSOD frameworks to boost their performance.