Search

Image decomposition aims to analyze an image into elementary components, which is essential for numerous downstream tasks and also by nature provides certain interpretability to the analysis. Deep learning can be powerful for such tasks, but surprisingly their combination with a focus on interpretability and generalizability is rarely explored. In this work, we introduce a novel framework for interpretable deep image decomposition, combining hierarchical Bayesian modeling and deep learning to create an architecture-modularized and model-generalizable deep neural network (DNN). The proposed framework includes three steps: (1) hierarchical Bayesian modeling of image decomposition, (2) transforming the inference problem into optimization tasks, and (3) deep inference via a modularized Bayesian DNN. We further establish a theoretical connection between the loss function and the generalization error bound, which inspires a new test-time adaptation approach for out-of-distribution scenarios. We instantiated the application using two downstream tasks, \textit{i.e.}, image denoising, and unsupervised anomaly detection, and the results demonstrated improved generalizability as well as interpretability of our methods. The source code will be released upon the acceptance of this paper.

This Letter reports the precise measurement of reactor antineutrino spectrum and flux based on the full data set of 4.7 million inverse-beta-decay (IBD) candidates collected at Daya Bay near detectors. Expressed in terms of the IBD yield per fission, the antineutrino spectra from all reactor fissile isotopes and the specific $\mathrm{^{235}U}$ and $\mathrm{^{239}Pu}$ isotopes are measured with 1.3$\%$, 3$\%$ and 8$\%$ uncertainties respectively near the 3 MeV spectrum peak in reconstructed energy, reaching the best precision in the world. The total antineutrino flux and isotopic $\mathrm{^{235}U}$ and $\mathrm{^{239}Pu}$ fluxes are precisely measured to be $5.84\pm0.07$, $6.16\pm0.12$ and $4.16\pm0.21$ in units of $10^{-43} \mathrm{cm^2/fission}$. These measurements are compared with the Huber-Mueller (HM) model, the reevaluated conversion model based on the Kurchatov Institute (KI) measurement and the latest Summation Model (SM2023). The Daya Bay flux shows good consistency with KI and SM2023 models, but disagrees with HM model. The Daya Bay spectrum, however, disagrees with all model predictions.

Missing attribute issues are prevalent in the graph learning, leading to biased outcomes in Graph Neural Networks (GNNs). Existing methods that rely on feature propagation are prone to cold start problem, particularly when dealing with attribute resetting and low-degree nodes, which hinder effective propagation and convergence. To address these challenges, we propose AttriReBoost (ARB), a novel method that incorporates propagation-based method to mitigate cold start problems in attribute-missing graphs. ARB enhances global feature propagation by redefining initial boundary conditions and strategically integrating virtual edges, thereby improving node connectivity and ensuring more stable and efficient convergence. This method facilitates gradient-free attribute reconstruction with lower computational overhead. The proposed method is theoretically grounded, with its convergence rigorously established. Extensive experiments on several real-world benchmark datasets demonstrate the effectiveness of ARB, achieving an average accuracy improvement of 5.11% over state-of-the-art methods. Additionally, ARB exhibits remarkable computational efficiency, processing a large-scale graph with 2.49 million nodes in just 16 seconds on a single GPU. Our code is available at https://github.com/limengran98/ARB.

Compared to image-text pair data, interleaved corpora enable Vision-Language Models (VLMs) to understand the world more naturally like humans. However, such existing datasets are crawled from webpage, facing challenges like low knowledge density, loose image-text relations, and poor logical coherence between images. On the other hand, the internet hosts vast instructional videos (e.g., online geometry courses) that are widely used by humans to learn foundational subjects, yet these valuable resources remain underexplored in VLM training. In this paper, we introduce a high-quality \textbf{multimodal textbook} corpus with richer foundational knowledge for VLM pretraining. It collects over 2.5 years of instructional videos, totaling 22,000 class hours. We first use an LLM-proposed taxonomy to systematically gather instructional videos. Then we progressively extract and refine visual (keyframes), audio (ASR), and textual knowledge (OCR) from the videos, and organize as an image-text interleaved corpus based on temporal order. Compared to its counterparts, our video-centric textbook offers more coherent context, richer knowledge, and better image-text alignment. Experiments demonstrate its superb pretraining performance, particularly in knowledge- and reasoning-intensive tasks like ScienceQA and MathVista. Moreover, VLMs pre-trained on our textbook exhibit outstanding interleaved context awareness, leveraging visual and textual cues in their few-shot context for task solving~\footnote{Our code are available at \url{https://github.com/DAMO-NLP-SG/multimodal_textbook}}., Comment: Under review

The superconducting diode effect (SDE) refers to the nonreciprocity of superconducting critical currents for the metal-superconductor transition. Generally, the SDE has a positive and a negative critical current $j_{c\pm}$ corresponding to two opposite directions whose amplitudes are unequal. It is demonstrated that an extreme nonreciprocity where two critical currents can become both positive (or negative) has been observed in a recent experiment. In this work, we theoretically propose a possible mechanism to realize an extreme nonreciprocal SDE. Based on a microscopic theory and a simple valley-polarized model, we demonstrate that depairing currents required to dissolve Cooper pairs can be remodulated under the interplay between the valley polarization and the applied current. Near the disappearance of the superconductivity, the remodulation is shown to induce the extreme nonreciprocity and also the current-induced re-entrant superconductivity where the system has two different critical current intervals. Our study may provide new horizons for understanding the coexistence of superconductivity and spontaneous ferromagnetism and pave a new way to designing the SDE with 100% efficiency., Comment: 14 pages, 8 figures, This manusrcipt was submitted to Phys. Rev. Lett. on December 4, 2023

Video Large Language Models (Video LLMs) have recently exhibited remarkable capabilities in general video understanding. However, they mainly focus on holistic comprehension and struggle with capturing fine-grained spatial and temporal details. Besides, the lack of high-quality object-level video instruction data and a comprehensive benchmark further hinders their advancements. To tackle these challenges, we introduce the VideoRefer Suite to empower Video LLM for finer-level spatial-temporal video understanding, i.e., enabling perception and reasoning on any objects throughout the video. Specially, we thoroughly develop VideoRefer Suite across three essential aspects: dataset, model, and benchmark. Firstly, we introduce a multi-agent data engine to meticulously curate a large-scale, high-quality object-level video instruction dataset, termed VideoRefer-700K. Next, we present the VideoRefer model, which equips a versatile spatial-temporal object encoder to capture precise regional and sequential representations. Finally, we meticulously create a VideoRefer-Bench to comprehensively assess the spatial-temporal understanding capability of a Video LLM, evaluating it across various aspects. Extensive experiments and analyses demonstrate that our VideoRefer model not only achieves promising performance on video referring benchmarks but also facilitates general video understanding capabilities., Comment: 17 pages, 12 figures, technical report

Built upon the previous work on the 4d anomalies and 5d cobordism invariants (namely, 5d invertible field theory [iTFT] or symmetry-protected topological state [SPTs]) of the Standard Model [SM] gauge theory with compatible (SU(3)$\times$SU(2)$\times$U(1))/$\mathbb{Z}_q$ gauge group for $q=1,2,3,6$, we further enumerate lower-dimensional iTFT / SPTs in 4d, 3d, 2d, and 1d. While the 4d SPTs are interesting gapped phases attachable to the SM, those integer classes of SPTs (either the torsion or the free cobordism class) are not universal and are difficult to detect. However, fractional SPTs are more universal and can be detected by a topological response similar to the Hall conductance under an appropriate symmetry twist background field. We find a nontrivial symmetry-extension from a magnetic 1-symmetry U(1)$_{[1]}^m$ by an electric 1-symmetry $\mathbb{Z}_{6/q,[1]}^e$ introducing the $k \in \mathbb{Z}_{6/q}$ fractionalization class. While we also introduce the background fields for Baryon minus Lepton number $({\bf B}-{\bf L})$ like 0-symmetries of U(1)$_{{\bf Q} - N_c {\bf L}}$, U(1)$_X$, and $\mathbb{Z}_{4,X}$, that introduces two more data $\alpha \in 1,3$ and $\beta \in 1,5$ for the SMs. We present their Hall conductance response data (denoted as $\sigma, \sigma'$, and $\sigma''$ respectively), between 0-symmetry and 1-symmetry background fields, and show their fractional dependence on the SM data $(q,\alpha,\beta, k)$., Comment: Sequel to: arXiv:1910.14668, arXiv:2112.14765, arXiv:2204.08393

Mobile edge computing (MEC) has empowered mobile devices (MDs) in supporting artificial intelligence (AI) applications through collaborative efforts with proximal MEC servers. Unfortunately, despite the great promise of device-edge cooperative AI inference, data privacy becomes an increasing concern. In this paper, we develop a privacy-aware multi-device cooperative edge inference system for classification tasks, which integrates a distributed bidding mechanism for the MEC server's computational resources. Intermediate feature compression is adopted as a principled approach to minimize data privacy leakage. To determine the bidding values and feature compression ratios in a distributed fashion, we formulate a decentralized partially observable Markov decision process (DEC-POMDP) model, for which, a multi-agent deep deterministic policy gradient (MADDPG)-based algorithm is developed. Simulation results demonstrate the effectiveness of the proposed algorithm in privacy-preserving cooperative edge inference. Specifically, given a sufficient level of data privacy protection, the proposed algorithm achieves 0.31-0.95% improvements in classification accuracy compared to the approach being agnostic to the wireless channel conditions. The performance is further enhanced by 1.54-1.67% by considering the difficulties of inference data., Comment: This article was submitted to IEEE for possible publication

Bayesian optimization is an effective technique for black-box optimization, but its applicability is typically limited to low-dimensional and small-budget problems due to the cubic complexity of computing the Gaussian process (GP) surrogate. While various approximate GP models have been employed to scale Bayesian optimization to larger sample sizes, most suffer from overly-smooth estimation and focus primarily on problems that allow for large online samples. In this work, we argue that Bayesian optimization algorithms with sparse GPs can more efficiently allocate their representational power to relevant regions of the search space. To achieve this, we propose focalized GP, which leverages a novel variational loss function to achieve stronger local prediction, as well as FocalBO, which hierarchically optimizes the focalized GP acquisition function over progressively smaller search spaces. Experimental results demonstrate that FocalBO can efficiently leverage large amounts of offline and online data to achieve state-of-the-art performance on robot morphology design and to control a 585-dimensional musculoskeletal system., Comment: Accepted by NeurIPS 2024

Multimodal large language models (MLLMs) enhance their perceptual capabilities by integrating visual and textual information. However, processing the massive number of visual tokens incurs a significant computational cost. Existing analysis of the MLLM attention mechanisms remains shallow, leading to coarse-grain token pruning strategies that fail to effectively balance speed and accuracy. In this paper, we conduct a comprehensive investigation of MLLM attention mechanisms with LLaVA. We find that numerous visual tokens and partial attention computations are redundant during the decoding process. Based on this insight, we propose Spatial-Temporal Visual Token Trimming ($\textbf{ST}^{3}$), a framework designed to accelerate MLLM inference without retraining. $\textbf{ST}^{3}$ consists of two primary components: 1) Progressive Visual Token Pruning (\textbf{PVTP}), which eliminates inattentive visual tokens across layers, and 2) Visual Token Annealing (\textbf{VTA}), which dynamically reduces the number of visual tokens in each layer as the generated tokens grow. Together, these techniques deliver around $\mathbf{2\times}$ faster inference with only about $\mathbf{30\%}$ KV cache memory compared to the original LLaVA, while maintaining consistent performance across various datasets. Crucially, $\textbf{ST}^{3}$ can be seamlessly integrated into existing pre-trained MLLMs, providing a plug-and-play solution for efficient inference., Comment: Accepted to AAAI2025

Recursion is the nature of human natural language. Since Chomsky proposed generative grammar, many scholars have studied recursion either theoretically or empirically. However, by observing children's acquisition of tail recursion sequences, we can verify the nativism of language supported by universal grammar and reveal the cognitive mechanism of human brain. To date, our understanding of children's acquisition path of recursion and influencing factors still remain controversial. This systematic review summarizes the research of tail recursive sequence by taking possessive recursion and locative recursion as examples, focusing on the experimental methods, acquisition paths, and influencing factors of tail recursive sequence. The current behavioural experiments reveal that, the debate about children's performance revolves around: 1) Gradual acquisition or synchronous acquisition. 2) symmetry or asymmetry between the acquisition of locative recursion sequences and possessive recursion sequences. We presume that children can acquire recursion quickly in a short period of time thanks to the language acquisition device, though there are also scholars who believe that a third factor also plays a role., Comment: 32 pages, 5 figures

Diffusion-based text-to-image (T2I) models have demonstrated remarkable results in global video editing tasks. However, their focus is primarily on global video modifications, and achieving desired attribute-specific changes remains a challenging task, specifically in multi-attribute editing (MAE) in video. Contemporary video editing approaches either require extensive fine-tuning or rely on additional networks (such as ControlNet) for modeling multi-object appearances, yet they remain in their infancy, offering only coarse-grained MAE solutions. In this paper, we present MAKIMA, a tuning-free MAE framework built upon pretrained T2I models for open-domain video editing. Our approach preserves video structure and appearance information by incorporating attention maps and features from the inversion process during denoising. To facilitate precise editing of multiple attributes, we introduce mask-guided attention modulation, enhancing correlations between spatially corresponding tokens and suppressing cross-attribute interference in both self-attention and cross-attention layers. To balance video frame generation quality and efficiency, we implement consistent feature propagation, which generates frame sequences by editing keyframes and propagating their features throughout the sequence. Extensive experiments demonstrate that MAKIMA outperforms existing baselines in open-domain multi-attribute video editing tasks, achieving superior results in both editing accuracy and temporal consistency while maintaining computational efficiency.

Optical frequency combs have emerged as a cornerstone for a wide range of areas, including spectroscopy, ranging, optical clocks, time and frequency transfer, waveform synthesis, and communications. However, quantum mechanical fluctuations of the optical carrier impose fundamental performance limits on the precision of traditional classical laser frequency combs, particularly in their use for interferometry and spectroscopy. Entanglement, as a quintessential quantum resource, allows for surpassing the fundamental limits of classical systems. Here, we introduce and experimentally demonstrate entangled dual-comb spectroscopy (EDCS) that surmounts the fundamental limits of classical DCS. EDCS builds on tailored entangled spectral structures of the frequency combs, enabling simultaneous detection of all comb lines below the standard quantum limit of classical DCS. Applying EDCS in gas detection, we achieve a 2.6 dB enhancement in signal-to-noise ratio and a 1.7-fold reduction in integration time over classical DCS, rendering EDCS particularly suited for dynamic chemical and biological sensing, where fast, precise measurements subject to power constraints are required. EDCS represents a new paradigm for quantum frequency combs, underscoring their prospects in a plethora of applications in precision metrology, spectroscopy, and timekeeping., Comment: 19 pages, 8 figures

Charge conjugation (C), mirror reflection (R), and time reversal (T) symmetries, along with internal symmetries, are essential for massless Majorana and Dirac fermions. These symmetries are sufficient to rule out potential fermion bilinear mass terms, thereby establishing a gapless free fermion fixed point phase, pivotal for symmetric mass generation (SMG) transition. In this work, we systematically study the anomaly of C-R-T-internal symmetry in all spacetime dimensions by analyzing the projective representation (i.e. the fractionalization) of the C-R-T-internal symmetry group in the quantum many-body Hilbert space on the lattice. By discovering the fermion-flavor-number-dependent C-R-T-internal symmetry's anomaly structure, we demonstrate an alternative way to derive the minimal flavor number for SMG, which shows consistency with known results from K\"ahler-Dirac fermion or cobordism classification. Our findings reveal that, in general spatial dimensions, either 8 copies of staggered Majorana fermions or 4 copies of staggered Dirac fermions admit SMG. By directly searching for 4-fermion interactions that form commuting stabilizers respecting all symmetry constraints, we can prove the explicit SMG gapping retained a unique ground state in the codespace. Furthermore, we establish the correspondence between the symmetry operators of staggered fermions and free fermions, which is instrumental in facilitating the analysis of symmetry fractionalization at the field theory level., Comment: 36 pages, 2 figures, 13 tables, sequel to arXiv:2109.15320, arXiv:2312.17126, and arXiv:2412.11958. Motivation: CRT-internal symmetry can rule out fermion bilinear mass terms. Key Result: Examine the fractionalization and anomalies of the CRT-internal symmetry on the lattice and find the minimum flavor number for SMG in all dimensions for both Majorana and Dirac fermions

Efficient multimodal large language models (EMLLMs), in contrast to multimodal large language models (MLLMs), reduce model size and computational costs and are often deployed on resource-constrained devices. However, due to data privacy concerns, existing open-source EMLLMs rarely have access to private domain-specific data during the pre-training process, making them difficult to directly apply in device-specific domains, such as certain business scenarios. To address this weakness, this paper focuses on the efficient adaptation of EMLLMs to private domains, specifically in two areas: 1) how to reduce data requirements, and 2) how to avoid parameter fine-tuning. Specifically, we propose a tun\textbf{\underline{I}}ng-free, a\textbf{\underline{D}}aptiv\textbf{\underline{E}}, univers\textbf{\underline{AL}} \textbf{\underline{Prompt}} Optimization Framework, abbreviated as \textit{\textbf{\ourmethod{}}} which consists of two stages: 1) Predefined Prompt, based on the reinforcement searching strategy, generate a prompt optimization strategy tree to acquire optimization priors; 2) Prompt Reflection initializes the prompt based on optimization priors, followed by self-reflection to further search and refine the prompt. By doing so, \ourmethod{} elegantly generates the ``ideal prompts'' for processing private domain-specific data. Note that our method requires no parameter fine-tuning and only a small amount of data to quickly adapt to the data distribution of private data. Extensive experiments across multiple tasks demonstrate that our proposed \ourmethod{} significantly improves both efficiency and performance compared to baselines.

We study the representation theory of the spherical double affine Hecke algebra (DAHA) of $C^\vee C_1$, using brane quantization. By showing a one-to-one correspondence between Lagrangian $A$-branes with compact support and finite-dimensional representations of the spherical DAHA, we provide evidence of derived equivalence between the $A$-brane category of $\mathrm{SL}(2,\mathbb{C})$-character variety of a four-punctured sphere and the representation category of DAHA of $C^\vee C_1$. The $D_4$ root system plays an essential role in understanding both the geometry and representation theory. In particular, this $A$-model approach reveals the action of an affine braid group of type $D_4$ on the category. As a by-product, our geometric investigation offers detailed information about the low-energy dynamics of the SU(2) $N_f=4$ Seiberg-Witten theory., Comment: 75 pages, 30 figures, 11 tables

Elastocaloric cooling with shape memory alloys (SMAs) is emerging as a promising candidate for next-generation, environmentally friendly refrigeration. However, its development is hindered by the large driving force and low efficiency associated with uniaxial loading modes. In response, we present an innovative elastocaloric air cooling approach that utilizes the bending of NiTi beams, offering a low-force and energy-efficient solution. We achieve a continuous maximum temperature drop of 0.91 K and a dissipated energy of 15.1 N mm at a low specific driving force of 220 N. Notably, the specimen achieves over 5 million cycles under a maximum surface tensile strain of 1.94% for macroscopic cyclic bending, via a pre-strained, warm laser shock peening (pw-LSP) method. This unprecedented fatigue resistance originates from the formation of a hierarchical microstructure and a large compressive residual stress of over 1 GPa. This work demonstrates the great potential of bending induced elastocaloric cooling in the near future., Comment: 75 pages, 41 figures

Ensuring safety and efficiency in emerging hydrogen-hydrocarbon fuel systems requires accurate measurement of multiple gas components in real time. However, existing detection techniques generally lack the capability to quantitatively measure hydrogen and natural gas constituents simultaneously. Here, we present a novel conductance-photoacoustic spectroscopy (CPAS) method that integrates platinum-modified conductance measurements with beat-frequency photoacoustic detection. By bridging a quartz tuning fork with a platinum microwire, our approach enables direct monitoring of hydrogen concentration via frequency modulation, while simultaneously capturing propane's photoacoustic signal with a single detection channel. Experimental results confirm that the platinum microwire effectively fine-tunes the tuning fork's mechanical properties for high-sensitivity hydrogen measurement, and the beat-frequency photoacoustic signals from propane absorption reveal complementary hydrocarbon concentration information. This unified sensor design is inherently compact, rapid, and calibration-free, making it particularly suitable for applications that demand real-time multiparameter gas analysis, including industrial process control and environmental monitoring. Taken together, these findings demonstrate that combining conductance and photoacoustic spectroscopies into a single integrated platform significantly advances the state of multi-component gas detection and holds promise for further enhancements in sensitivity and adaptability.

Large language models (LLMs) have transformed natural language processing, with frameworks like Chatbot Arena providing pioneering platforms for evaluating these models. By facilitating millions of pairwise comparisons based on human judgments, Chatbot Arena has become a cornerstone in LLM evaluation, offering rich datasets for ranking models in open-ended conversational tasks. Building upon this foundation, we propose a statistical framework that incorporates key advancements to address specific challenges in pairwise comparison analysis. First, we introduce a factored tie model that enhances the ability to handle ties -- an integral aspect of human-judged comparisons -- significantly improving the model's fit to observed data. Second, we extend the framework to model covariance between competitors, enabling deeper insights into performance relationships and facilitating intuitive groupings into performance tiers. Third, we resolve optimization challenges arising from parameter non-uniqueness by introducing novel constraints, ensuring stable and interpretable parameter estimation. Through rigorous evaluation and extensive experimentation, our framework demonstrates substantial improvements over existing methods in modeling pairwise comparison data. To support reproducibility and practical adoption, we release leaderbot, an open-source Python package implementing our models and analyses.

In this article, we address the metrological problem of estimating collective stochastic properties of a many-body quantum system. Canonical examples include center-of-mass quadrature fluctuations in a system of bosonic modes and correlated dephasing in an ensemble of qubits (e.g., spins) or fermions. We develop a theoretical framework to determine the limits of correlated (weak) noise estimation with quantum sensor networks and unveil the requirements for entanglement advantage. Notably, an advantage emerges from the synergistic interplay between quantum correlations of the sensors and classical (spatial) correlations of the noises. We determine optimal entangled probe states and identify a sensing protocol, reminiscent of a many-body echo sequence, that achieves the fundamental limits of measurement sensitivity for a broad class of problems., Comment: 7+6 pages, 1 figure. Feedback welcome!

Objectives: Prior event rate ratio (PERR) is a method shown to perform well in mitigating confounding in real-world evidence research but it depends on several model assumptions. We propose an analytic strategy to correct biases arising from violation of two model assumptions, namely, population homogeneity and event-independent treatment. Study Design and Setting: We reformulate PERR estimation by embedding a treatment-by-period interaction term in an analytic model for recurrent event data, which is robust to bias arising from unobserved heterogeneity. Based on this model, we propose a set of methods to examine the presence of event-dependent treatment and to correct the resultant bias. We evaluate the proposed methods by simulation and apply it to a de-identified dataset on palliative care and emergency department visits in patients with advanced cancer. Results: Simulation results showed that the proposed method could mitigate the two sources of bias in PERR. In the palliative care study, analysis by the Cox model showed that patients who had started receiving palliative care had higher incidence of emergency department visits than their match controls (hazard ratio 3.31; 95% confidence interval 2.78 to 3.94). Using PERR without the proposed bias control strategy indicated a 19% reduction of the incidence (0.81; 0.64 to 1.02). However, there was evidence of event-dependent treatment. The proposed correction method showed no effect of palliative care on ED visits (1.00; 0.79 to 1.26). Conclusions: The proposed analytic strategy can control two sources of biases in the PERR approach. It enriches the armamentarium for real-world evidence research., Comment: 35 pages, including 3 tables, 1 figure and 3 supplemental materials

In this study, we propose a data-driven, deep-learning-based Machine-Learning Symmetry Discovery (MLSD) algorithm to automate the discovery of continuous Lie group symmetries in classical mechanical systems from their time-evolution trajectory data. MLSD uses neural networks (NNs) to predict conserved physical quantities that implement symmetry transformations of the phase space coordinates. After training, MLSD is able to identify the Lie algebra, particularly non-abelian ones, as indicated by the Lie algebra structure coefficients. To demonstrate the effectiveness of the MLSD method, we applied it to simulated data from the classical three-dimensional Kepler problem and the harmonic oscillator. The results show that the algorithm successfully identified the hidden symmetry groups of both systems., Comment: 6 pages + references, 8 figures

Creating a high-fidelity, animatable 3D full-body avatar from a single image is a challenging task due to the diverse appearance and poses of humans and the limited availability of high-quality training data. To achieve fast and high-quality human reconstruction, this work rethinks the task from the perspectives of dataset, model, and representation. First, we introduce a large-scale HUman-centric GEnerated dataset, HuGe100K, consisting of 100K diverse, photorealistic sets of human images. Each set contains 24-view frames in specific human poses, generated using a pose-controllable image-to-multi-view model. Next, leveraging the diversity in views, poses, and appearances within HuGe100K, we develop a scalable feed-forward transformer model to predict a 3D human Gaussian representation in a uniform space from a given human image. This model is trained to disentangle human pose, body shape, clothing geometry, and texture. The estimated Gaussians can be animated without post-processing. We conduct comprehensive experiments to validate the effectiveness of the proposed dataset and method. Our model demonstrates the ability to efficiently reconstruct photorealistic humans at 1K resolution from a single input image using a single GPU instantly. Additionally, it seamlessly supports various applications, as well as shape and texture editing tasks., Comment: 21 pages, 15 figures, includes main content, supplementary materials, and references

The ALTA shared tasks have been running annually since 2010. In 2024, the purpose of the task is to detect machine-generated text in a hybrid setting where the text may contain portions of human text and portions machine-generated. In this paper, we present the task, the evaluation criteria, and the results of the systems participating in the shared task., Comment: 6 pages, 3 tables, published in ALTA 2024

The recent advent of large-scale 3D data, e.g. Objaverse, has led to impressive progress in training pose-conditioned diffusion models for novel view synthesis. However, due to the synthetic nature of such 3D data, their performance drops significantly when applied to real-world images. This paper consolidates a set of good practices to finetune large pretrained models for a real-world task -- harvesting vehicle assets for autonomous driving applications. To this end, we delve into the discrepancies between the synthetic data and real driving data, then develop several strategies to account for them properly. Specifically, we start with a virtual camera rotation of real images to ensure geometric alignment with synthetic data and consistency with the pose manifold defined by pretrained models. We also identify important design choices in object-centric data curation to account for varying object distances in real driving scenes -- learn across varying object scales with fixed camera focal length. Further, we perform occlusion-aware training in latent spaces to account for ubiquitous occlusions in real data, and handle large viewpoint changes by leveraging a symmetric prior. Our insights lead to effective finetuning that results in a $68.8\%$ reduction in FID for novel view synthesis over prior arts.

Generation with source attribution is important for enhancing the verifiability of retrieval-augmented generation (RAG) systems. However, existing approaches in RAG primarily link generated content to document-level references, making it challenging for users to locate evidence among multiple content-rich retrieved documents. To address this challenge, we propose Retrieval-Augmented Generation with Visual Source Attribution (VISA), a novel approach that combines answer generation with visual source attribution. Leveraging large vision-language models (VLMs), VISA identifies the evidence and highlights the exact regions that support the generated answers with bounding boxes in the retrieved document screenshots. To evaluate its effectiveness, we curated two datasets: Wiki-VISA, based on crawled Wikipedia webpage screenshots, and Paper-VISA, derived from PubLayNet and tailored to the medical domain. Experimental results demonstrate the effectiveness of VISA for visual source attribution on documents' original look, as well as highlighting the challenges for improvement. Code, data, and model checkpoints will be released.

``Little Red Dots'' (LRDs) are an abundant high-redshift population newly discovered by the James Webb Space Telescope (JWST). They are characterized by a red color in the rest-frame optical band, compact morphology, and broad Balmer emission lines (${\rm FWHM} \gtrsim 1000~{\rm km\,s^{-1}}$) that suggest an AGN nature. Using a method of pixel-by-pixel color selection and relaxing the compactness criteria, we identify three of the first dual LRD candidates in the COSMOS-Web survey with projected separations of $0.\!\!^{\prime\prime}2-0.\!\!^{\prime\prime}4$ (1-2 pkpc at their photometric redshifts). A comparison between existing LRD samples and mock data reveals that the projected separations of these dual LRD candidates are unlikely to result from chance projections of objects at different redshifts. In one case (CW-B5-15958), the dual LRD includes two bright sources ($m_{\rm F444W}=24.3$ and $24.8$) with characteristic V-shape spectral energy distribution (SEDs) and photometric redshifts consistent with each other. We find that CW-B5-15958 has a faint off-centered component and a companion galaxy. In the other two dual systems, the brighter LRD exhibits a V-shape SED, while the fainter LRD ($m_{\rm F444W}\gtrsim26$) is undetected in both F115W and F150W. These discoveries suggest that the angular auto-correlation function (ACF) of LRDs exhibits a significant excess ($\sim3\times10^2$ times) on sub-arcsec (kilo-parsec) separations compared to the extrapolation of a power-law ACF of JWST-found AGNs measured over $10^{\prime\prime}-100^{\prime\prime}$. Follow-up spectroscopic confirmation of their redshifts and the construction of a larger sample are essential to advance our understanding of the evolution of supermassive black holes and the importance of mergers in the early universe., Comment: 12 pages, 5 figures, and 1 table. Comments are welcome

In this work, we propose Visual-Predictive Instruction Tuning (VPiT) - a simple and effective extension to visual instruction tuning that enables a pretrained LLM to quickly morph into an unified autoregressive model capable of generating both text and visual tokens. VPiT teaches an LLM to predict discrete text tokens and continuous visual tokens from any input sequence of image and text data curated in an instruction-following format. Our empirical investigation reveals several intriguing properties of VPiT: (1) visual generation ability emerges as a natural byproduct of improved visual understanding, and can be unlocked efficiently with a small amount of generation data; (2) while we find understanding and generation to be mutually beneficial, understanding data contributes to both capabilities more effectively than generation data. Building upon these findings, we train our MetaMorph model and achieve competitive performance on both visual understanding and generation. In visual generation, MetaMorph can leverage the world knowledge and reasoning abilities gained from LLM pretraining, and overcome common failure modes exhibited by other generation models. Our results suggest that LLMs may have strong "prior" vision capabilities that can be efficiently adapted to both visual understanding and generation with a relatively simple instruction tuning process., Comment: Project page at tsb0601.github.io/metamorph

Despite the remarkable capabilities of large language models (LLMs) in natural language understanding and reasoning, they often display undesirable behaviors, such as generating hallucinations and unfaithful reasoning. A prevalent strategy to mitigate these issues is the use of reflection, which refines responses through an iterative process. However, while promising, reflection heavily relies on high-quality external feedback and requires iterative multi-agent inference processes, thus hindering its practical application. In this paper, we propose Meta-Reflection, a novel feedback-free reflection mechanism that necessitates only a single inference pass without external feedback. Motivated by the human ability to remember and retrieve reflections from past experiences when encountering similar problems, Meta-Reflection integrates reflective insights into a codebook, allowing the historical insights to be stored, retrieved, and used to guide LLMs in problem-solving. To thoroughly investigate and evaluate the practicality of Meta-Reflection in real-world scenarios, we introduce an industrial e-commerce benchmark named E-commerce Customer Intent Detection (ECID). Extensive experiments conducted on both public datasets and the ECID benchmark highlight the effectiveness and efficiency of our proposed approach.

Domain generalization aims to learn a model from multiple training domains and generalize it to unseen test domains. Recent theory has shown that seeking the deep models, whose parameters lie in the flat minima of the loss landscape, can significantly reduce the out-of-domain generalization error. However, existing methods often neglect the consistency of loss landscapes in different domains, resulting in models that are not simultaneously in the optimal flat minima in all domains, which limits their generalization ability. To address this issue, this paper proposes an iterative Self-Feedback Training (SFT) framework to seek consistent flat minima that are shared across different domains by progressively refining loss landscapes during training. It alternatively generates a feedback signal by measuring the inconsistency of loss landscapes in different domains and refines these loss landscapes for greater consistency using this feedback signal. Benefiting from the consistency of the flat minima within these refined loss landscapes, our SFT helps achieve better out-of-domain generalization. Extensive experiments on DomainBed demonstrate superior performances of SFT when compared to state-of-the-art sharpness-aware methods and other prevalent DG baselines. On average across five DG benchmarks, SFT surpasses the sharpness-aware minimization by 2.6% with ResNet-50 and 1.5% with ViT-B/16, respectively. The code will be available soon.

Recent studies have indicated that effectively utilizing inference-time compute is crucial for attaining better performance from large language models (LLMs). In this work, we propose a novel inference-aware fine-tuning paradigm, in which the model is fine-tuned in a manner that directly optimizes the performance of the inference-time strategy. We study this paradigm using the simple yet effective Best-of-N (BoN) inference strategy, in which a verifier selects the best out of a set of LLM-generated responses. We devise the first imitation learning and reinforcement learning~(RL) methods for BoN-aware fine-tuning, overcoming the challenging, non-differentiable argmax operator within BoN. We empirically demonstrate that our BoN-aware models implicitly learn a meta-strategy that interleaves best responses with more diverse responses that might be better suited to a test-time input -- a process reminiscent of the exploration-exploitation trade-off in RL. Our experiments demonstrate the effectiveness of BoN-aware fine-tuning in terms of improved performance and inference-time compute. In particular, we show that our methods improve the Bo32 performance of Gemma 2B on Hendrycks MATH from 26.8% to 30.8%, and pass@32 from 60.0% to 67.0%, as well as the pass@16 on HumanEval from 61.6% to 67.1%.

Lately, the practice of utilizing task-specific fine-tuning has been implemented to improve the performance of large language models (LLM) in subsequent tasks. Through the integration of diverse LLMs, the overall competency of LLMs is significantly boosted. Nevertheless, traditional ensemble methods are notably memory-intensive, necessitating the simultaneous loading of all specialized models into GPU memory. To address the inefficiency, model merging strategies have emerged, merging all LLMs into one model to reduce the memory footprint during inference. Despite these advances, model merging often leads to parameter conflicts and performance decline as the number of experts increases. Previous methods to mitigate these conflicts include post-pruning and partial merging. However, both approaches have limitations, particularly in terms of performance and storage efficiency when merged experts increase. To address these challenges, we introduce Channel Merging, a novel strategy designed to minimize parameter conflicts while enhancing storage efficiency. This method clusters and merges channel parameters based on their similarity to form several groups offline. By ensuring that only highly similar parameters are merged within each group, it significantly reduces parameter conflicts. During inference, we can instantly look up the expert parameters from the merged groups, preserving specialized knowledge. Our experiments demonstrate that Channel Merging consistently delivers high performance, matching unmerged models in tasks like English and Chinese reasoning, mathematical reasoning, and code generation. Moreover, it obtains results comparable to model ensemble with just 53% parameters when used with a task-specific router., Comment: accepted by AAAI 2025

The tearing mode, a large-scale MHD instability in tokamak, typically disrupts the equilibrium magnetic surfaces, leads to the formation of magnetic islands, and reduces core electron temperature and density, thus resulting in significant energy losses and may even cause discharge termination. This process is unacceptable for ITER. Therefore, the accurate identification of a magnetic island in real time is crucial for the effective control of the tearing mode in ITER in the future. In this study, based on the characteristics induced by tearing modes, an attention-aware convolutional neural network (AM-CNN) is proposed to identify the presence of magnetic islands in tearing mode discharge utilizing the data from ECE diagnostics in the EAST tokamak. A total of 11 ECE channels covering the range of core is used in the tearing mode dataset, which includes 2.5*10^9 data collected from 68 shots from 2016 to 2021 years. We split the dataset into training, validation, and test sets (66.5%, 5.7%, and 27.8%), respectively. An attention mechanism is designed to couple with the convolutional neural networks to improve the capability of feature extraction of signals. During the model training process, we utilized adaptive learning rate adjustment and early stopping mechanisms to optimize performance of AM-CNN. The model results show that a classification accuracy of 91.96% is achieved in tearing mode identification. Compared to CNN without AM, the attention-aware convolutional neural networks demonstrate great performance across accuracy, recall metrics, and F1 score. By leveraging the deep learning model, which incorporates a physical understanding of the tearing process to identify tearing mode behaviors, the combination of physical mechanisms and deep learning is emphasized, significantly laying an important foundation for the future intelligent control of tearing mode dynamics.

Massive Dirac fermions, which are essential for realizing novel topological phenomena, are expected to be generated from massless Dirac fermions by breaking the related symmetry, such as time-reversal symmetry (TRS) in topological insulators or crystal symmetry in topological crystalline insulators. Here, we report scanning tunneling microscopy and angle-resolved photoemission spectroscopy studies of {\alpha}-Bi4I4, which reveals the realization of massive Dirac fermions in the (100) surface states without breaking the TRS. Combined with first-principle calculations, our experimental results indicate that the spontaneous symmetry breaking engenders two nondegenerate edges states at the opposite sides of monolayer Bi4I4 after the structural phase transition, imparting mass to the Dirac fermions after taking the interlayer coupling into account. Our results not only demonstrate the formation of the massive Dirac fermions by spontaneous symmetry breaking, but also imply the potential for the engineering of Dirac fermions for device applications.

Using $(2712.4\pm14.3)\times10^{6}$ $\psi(3686)$ events collected with the BESIII detector at the BEPCII collider, the decay $\eta_c\to\gamma\gamma$ in $J/\psi\to\gamma\eta_c$ is observed for the first time. We determine the product branching fraction $\mathcal{B}(J/\psi\to\gamma\eta_c)\times\mathcal{B}(\eta_c\to\gamma\gamma)=(5.23\pm0.26_{\rm{stat.}}\pm0.30_{\rm{syst.}})\times10^{-6}$. This result is well consistent with the LQCD calculation $(5.34\pm0.16)\times10^{-6}$ from HPQCD in 2023. By using the world-average values of $\mathcal{B}(J/\psi\to\gamma\eta_c)$ and the total decay width of $\eta_c$, the partial decay width $\Gamma(\eta_c\to\gamma\gamma)$ is determined to be $(11.30\pm0.56_{\rm{stat.}}\pm0.66_{\rm{syst.}}\pm1.14_{\rm{ref.}})~\rm{keV}$, which deviates from the corresponding world-average value by $3.4\sigma$., Comment: 10 pages, 4 figures

In this paper, we propose a novel covariance information-assisted channel state information (CSI) feedback scheme for frequency-division duplex (FDD) massive multi-input multi-output (MIMO) systems. Unlike most existing CSI feedback schemes, which rely on instantaneous CSI only, the proposed CovNet leverages CSI covariance information to achieve high-performance CSI reconstruction, primarily consisting of convolutional neural network (CNN) and Transformer architecture. To efficiently utilize covariance information, we propose a covariance information processing procedure and sophisticatedly design the covariance information processing network (CIPN) to further process it. Moreover, the feed-forward network (FFN) in CovNet is designed to jointly leverage the 2D characteristics of the CSI matrix in the angle and delay domains. Simulation results demonstrate that the proposed network effectively leverages covariance information and outperforms the state-of-the-art (SOTA) scheme across the full compression ratio (CR) range., Comment: This work has been submitted to the IEEE for possible publication

The deployment of large language models (LLMs) in diverse applications requires a thorough understanding of their decision-making strategies and behavioral patterns. As a supplement to a recent study on the behavioral Turing test, this paper presents a comprehensive analysis of five leading LLM-based chatbot families as they navigate a series of behavioral economics games. By benchmarking these AI chatbots, we aim to uncover and document both common and distinct behavioral patterns across a range of scenarios. The findings provide valuable insights into the strategic preferences of each LLM, highlighting potential implications for their deployment in critical decision-making roles., Comment: Presented at The First Workshop on AI Behavioral Science (AIBS 2024)

Recent research has revealed that the CRT symmetry for fermions exhibits a fractionalization distinct from the $\mathbb{Z}_2^{\mathcal{C}}\times\mathbb{Z}_2^{\mathcal{R}}\times\mathbb{Z}_2^{\mathcal{T}}$ for scalar bosons. In fact, the CRT symmetry for fermions can be extended by internal symmetries such as fermion parity, thereby forming a group extension of the $\mathbb{Z}_2$ direct product. Conventionally, a Majorana fermion is defined by one Dirac fermion with trivial charge conjugation. However, when the spacetime dimension $d+1=5,6,7\bmod8$, the real dimension of Majorana fermion (dim$_{\mathbb{R}}\chi_{\mathcal{C}\ell(d,0)}$) aligns with the real dimension of Dirac fermion (dim$_{\mathbb{R}}\psi_{\mathcal{C}\ell(d)}$), rather than being half, which necessitates the introduction of a symplectic Majorana fermion, defined by two Dirac fermions with trivial charge conjugation. To include these two types of Majorana fermions, we embed the theory in $n_{\mathbb{R}}$ and define the Majorana fermion field as a representation of the real Clifford algebra with 8-fold periodicity. Within the Hamiltonian formalism, we identify the 8-fold CRT-internal symmetry groups across general dimensions. Similarly, Dirac fermion field is defined as a representation of the complex Clifford algebra with 2-fold periodicity. Interestingly, we discover that the CRT-internal symmetry groups exhibit an 8-fold periodicity that is distinct from that of the complex Clifford algebra. In certain dimensions where distinct mass terms can span a mass manifold, the CRT-internal symmetries can act non-trivially upon this mass manifold. Employing domain wall reduction method, we are able to elucidate the relationships between symmetries across different dimensions., Comment: 54 pages, 39 tables, sequel to arXiv:2109.15320 and arXiv:2312.17126. Include symplectic Majorana in d+1=5,6,7 mod 8, and symplectic Majorana-Weyl in d+1=6 mod 8. Motivation: use CRT-internal symmetries to rule out fermion bilinear masses. Key result: systematically assigned CRT-internal symmetries in all spacetime dimensions for both Majorana and Dirac fermions

Character-based dialogue (aka role-playing) enables users to freely customize characters for interaction, which often relies on LLMs, raising the need to evaluate LLMs' character customization capability. However, existing benchmarks fail to ensure a robust evaluation as they often only involve a single character category or evaluate limited dimensions. Moreover, the sparsity of character features in responses makes feature-focused generative evaluation both ineffective and inefficient. To address these issues, we propose CharacterBench, the largest bilingual generative benchmark, with 22,859 human-annotated samples covering 3,956 characters from 25 detailed character categories. We define 11 dimensions of 6 aspects, classified as sparse and dense dimensions based on whether character features evaluated by specific dimensions manifest in each response. We enable effective and efficient evaluation by crafting tailored queries for each dimension to induce characters' responses related to specific dimensions. Further, we develop CharacterJudge model for cost-effective and stable evaluations. Experiments show its superiority over SOTA automatic judges (e.g., GPT-4) and our benchmark's potential to optimize LLMs' character customization. Our repository is at https://github.com/thu-coai/CharacterBench., Comment: AAAI 2025

Multi-domain recommendation (MDR) aims to enhance recommendation performance across various domains. However, real-world recommender systems in online platforms often need to handle dozens or even hundreds of domains, far exceeding the capabilities of traditional MDR algorithms, which typically focus on fewer than five domains. Key challenges include a substantial increase in parameter count, high maintenance costs, and intricate knowledge transfer patterns across domains. Furthermore, minor domains often suffer from data sparsity, leading to inadequate training in classical methods. To address these issues, we propose Adaptive REcommendation for All Domains with counterfactual augmentation (AREAD). AREAD employs a hierarchical structure with a limited number of expert networks at several layers, to effectively capture domain knowledge at different granularities. To adaptively capture the knowledge transfer pattern across domains, we generate and iteratively prune a hierarchical expert network selection mask for each domain during training. Additionally, counterfactual assumptions are used to augment data in minor domains, supporting their iterative mask pruning. Our experiments on two public datasets, each encompassing over twenty domains, demonstrate AREAD's effectiveness, especially in data-sparse domains. Source code is available at https://github.com/Chrissie-Law/AREAD-Multi-Domain-Recommendation., Comment: Extended version accepted by AAAI 2025

Automatic black-and-white image sequence colorization while preserving character and object identity (ID) is a complex task with significant market demand, such as in cartoon or comic series colorization. Despite advancements in visual colorization using large-scale generative models like diffusion models, challenges with controllability and identity consistency persist, making current solutions unsuitable for industrial application.To address this, we propose ColorFlow, a three-stage diffusion-based framework tailored for image sequence colorization in industrial applications. Unlike existing methods that require per-ID finetuning or explicit ID embedding extraction, we propose a novel robust and generalizable Retrieval Augmented Colorization pipeline for colorizing images with relevant color references. Our pipeline also features a dual-branch design: one branch for color identity extraction and the other for colorization, leveraging the strengths of diffusion models. We utilize the self-attention mechanism in diffusion models for strong in-context learning and color identity matching. To evaluate our model, we introduce ColorFlow-Bench, a comprehensive benchmark for reference-based colorization. Results show that ColorFlow outperforms existing models across multiple metrics, setting a new standard in sequential image colorization and potentially benefiting the art industry. We release our codes and models on our project page: https://zhuang2002.github.io/ColorFlow/., Comment: Project Page: https://zhuang2002.github.io/ColorFlow/

The emergence of long-context text applications utilizing large language models (LLMs) has presented significant scalability challenges, particularly in memory footprint. The linear growth of the Key-Value (KV) cache responsible for storing attention keys and values to minimize redundant computations can lead to substantial increases in memory consumption, potentially causing models to fail to serve with limited memory resources. To address this issue, we propose a novel approach called Cache Sparse Representation (CSR), which converts the KV cache by transforming the dense Key-Value cache tensor into sparse indexes and weights, offering a more memory-efficient representation during LLM inference. Furthermore, we introduce NeuralDict, a novel neural network-based method for automatically generating the dictionary used in our sparse representation. Our extensive experiments demonstrate that CSR achieves performance comparable to state-of-the-art KV cache quantization algorithms while maintaining robust functionality in memory-constrained environments.

Unsupervised Graph Domain Adaptation (UGDA) seeks to bridge distribution shifts between domains by transferring knowledge from labeled source graphs to given unlabeled target graphs. Existing UGDA methods primarily focus on aligning features in the latent space learned by graph neural networks (GNNs) across domains, often overlooking structural shifts, resulting in limited effectiveness when addressing structurally complex transfer scenarios. Given the sensitivity of GNNs to local structural features, even slight discrepancies between source and target graphs could lead to significant shifts in node embeddings, thereby reducing the effectiveness of knowledge transfer. To address this issue, we introduce a novel approach for UGDA called Target-Domain Structural Smoothing (TDSS). TDSS is a simple and effective method designed to perform structural smoothing directly on the target graph, thereby mitigating structural distribution shifts and ensuring the consistency of node representations. Specifically, by integrating smoothing techniques with neighborhood sampling, TDSS maintains the structural coherence of the target graph while mitigating the risk of over-smoothing. Our theoretical analysis shows that TDSS effectively reduces target risk by improving model smoothness. Empirical results on three real-world datasets demonstrate that TDSS outperforms recent state-of-the-art baselines, achieving significant improvements across six transfer scenarios. The code is available in https://github.com/cwei01/TDSS., Comment: 11 pages, Accpected by AAAI2025

Human experts typically integrate numerical and textual multimodal information to analyze time series. However, most traditional deep learning predictors rely solely on unimodal numerical data, using a fixed-length window for training and prediction on a single dataset, and cannot adapt to different scenarios. The powered pre-trained large language model has introduced new opportunities for time series analysis. Yet, existing methods are either inefficient in training, incapable of handling textual information, or lack zero-shot forecasting capability. In this paper, we innovatively model time series as a foreign language and construct ChatTime, a unified framework for time series and text processing. As an out-of-the-box multimodal time series foundation model, ChatTime provides zero-shot forecasting capability and supports bimodal input/output for both time series and text. We design a series of experiments to verify the superior performance of ChatTime across multiple tasks and scenarios, and create four multimodal datasets to address data gaps. The experimental results demonstrate the potential and utility of ChatTime., Comment: Accepted by AAAI 2025

Recent diffusion models have demonstrated remarkable performance in both 3D scene generation and perception tasks. Nevertheless, existing methods typically separate these two processes, acting as a data augmenter to generate synthetic data for downstream perception tasks. In this work, we propose OccScene, a novel mutual learning paradigm that integrates fine-grained 3D perception and high-quality generation in a unified framework, achieving a cross-task win-win effect. OccScene generates new and consistent 3D realistic scenes only depending on text prompts, guided with semantic occupancy in a joint-training diffusion framework. To align the occupancy with the diffusion latent, a Mamba-based Dual Alignment module is introduced to incorporate fine-grained semantics and geometry as perception priors. Within OccScene, the perception module can be effectively improved with customized and diverse generated scenes, while the perception priors in return enhance the generation performance for mutual benefits. Extensive experiments show that OccScene achieves realistic 3D scene generation in broad indoor and outdoor scenarios, while concurrently boosting the perception models to achieve substantial performance improvements in the 3D perception task of semantic occupancy prediction.

While deep learning has made remarkable progress in recent years, models continue to struggle with catastrophic forgetting when processing continuously incoming data. This issue is particularly critical in continual learning, where the balance between retaining prior knowledge and adapting to new information-known as the stability-plasticity dilemma-remains a significant challenge. In this paper, we propose SegACIL, a novel continual learning method for semantic segmentation based on a linear closed-form solution. Unlike traditional methods that require multiple epochs for training, SegACIL only requires a single epoch, significantly reducing computational costs. Furthermore, we provide a theoretical analysis demonstrating that SegACIL achieves performance on par with joint learning, effectively retaining knowledge from previous data which makes it to keep both stability and plasticity at the same time. Extensive experiments on the Pascal VOC2012 dataset show that SegACIL achieves superior performance in the sequential, disjoint, and overlap settings, offering a robust solution to the challenges of class-incremental semantic segmentation. Code is available at https://github.com/qwrawq/SegACIL.

In recent years, as robotics has advanced, human-robot collaboration has gained increasing importance. However, current robots struggle to fully and accurately interpret human intentions from voice commands alone. Traditional gripper and suction systems often fail to interact naturally with humans, lack advanced manipulation capabilities, and are not adaptable to diverse tasks, especially in unstructured environments. This paper introduces the Embodied Dexterous Grasping System (EDGS), designed to tackle object grasping in cluttered environments for human-robot interaction. We propose a novel approach to semantic-object alignment using a Vision-Language Model (VLM) that fuses voice commands and visual information, significantly enhancing the alignment of multi-dimensional attributes of target objects in complex scenarios. Inspired by human hand-object interactions, we develop a robust, precise, and efficient grasping strategy, incorporating principles like the thumb-object axis, multi-finger wrapping, and fingertip interaction with an object's contact mechanics. We also design experiments to assess Referring Expression Representation Enrichment (RERE) in referring expression segmentation, demonstrating that our system accurately detects and matches referring expressions. Extensive experiments confirm that EDGS can effectively handle complex grasping tasks, achieving stability and high success rates, highlighting its potential for further development in the field of Embodied AI.

Significant disparities between the features of natural images and those inherent to histopathological images make it challenging to directly apply and transfer pre-trained models from natural images to histopathology tasks. Moreover, the frequent lack of annotations in histopathology patch images has driven researchers to explore self-supervised learning methods like mask reconstruction for learning representations from large amounts of unlabeled data. Crucially, previous mask-based efforts in self-supervised learning have often overlooked the spatial interactions among entities, which are essential for constructing accurate representations of pathological entities. To address these challenges, constructing graphs of entities is a promising approach. In addition, the diffusion reconstruction strategy has recently shown superior performance through its random intensity noise addition technique to enhance the robust learned representation. Therefore, we introduce H-MGDM, a novel self-supervised Histopathology image representation learning method through the Dynamic Entity-Masked Graph Diffusion Model. Specifically, we propose to use complementary subgraphs as latent diffusion conditions and self-supervised targets respectively during pre-training. We note that the graph can embed entities' topological relationships and enhance representation. Dynamic conditions and targets can improve pathological fine reconstruction. Our model has conducted pretraining experiments on three large histopathological datasets. The advanced predictive performance and interpretability of H-MGDM are clearly evaluated on comprehensive downstream tasks such as classification and survival analysis on six datasets. Our code will be publicly available at https://github.com/centurion-crawler/H-MGDM.

Digital agents are increasingly employed to automate tasks in interactive digital environments such as web pages, software applications, and operating systems. While text-based agents built on Large Language Models (LLMs) often require frequent updates due to platform-specific APIs, visual agents leveraging Multimodal Large Language Models (MLLMs) offer enhanced adaptability by interacting directly with Graphical User Interfaces (GUIs). However, these agents face significant challenges in visual perception, particularly when handling high-resolution, visually complex digital environments. This paper introduces Iris, a foundational visual agent that addresses these challenges through two key innovations: Information-Sensitive Cropping (ISC) and Self-Refining Dual Learning (SRDL). ISC dynamically identifies and prioritizes visually dense regions using a edge detection algorithm, enabling efficient processing by allocating more computational resources to areas with higher information density. SRDL enhances the agent's ability to handle complex tasks by leveraging a dual-learning loop, where improvements in referring (describing UI elements) reinforce grounding (locating elements) and vice versa, all without requiring additional annotated data. Empirical evaluations demonstrate that Iris achieves state-of-the-art performance across multiple benchmarks with only 850K GUI annotations, outperforming methods using 10x more training data. These improvements further translate to significant gains in both web and OS agent downstream tasks.

Human-object contact (HOT) is designed to accurately identify the areas where humans and objects come into contact. Current methods frequently fail to account for scenarios where objects are frequently blocking the view, resulting in inaccurate identification of contact areas. To tackle this problem, we suggest using a perspective interaction HOT detector called PIHOT, which utilizes a depth map generation model to offer depth information of humans and objects related to the camera, thereby preventing false interaction detection. Furthermore, we use mask dilatation and object restoration techniques to restore the texture details in covered areas, improve the boundaries between objects, and enhance the perception of humans interacting with objects. Moreover, a spatial awareness perception is intended to concentrate on the characteristic features close to the points of contact. The experimental results show that the PIHOT algorithm achieves state-of-the-art performance on three benchmark datasets for HOT detection tasks. Compared to the most recent DHOT, our method enjoys an average improvement of 13%, 27.5%, 16%, and 18.5% on SC-Acc., C-Acc., mIoU, and wIoU metrics, respectively., Comment: Accepted by AAAl 2025