Search

Generalizable 3D Gaussian splitting (3DGS) can reconstruct new scenes from sparse-view observations in a feed-forward inference manner, eliminating the need for scene-specific retraining required in conventional 3DGS. However, existing methods rely heavily on epipolar priors, which can be unreliable in complex realworld scenes, particularly in non-overlapping and occluded regions. In this paper, we propose eFreeSplat, an efficient feed-forward 3DGS-based model for generalizable novel view synthesis that operates independently of epipolar line constraints. To enhance multiview feature extraction with 3D perception, we employ a selfsupervised Vision Transformer (ViT) with cross-view completion pre-training on large-scale datasets. Additionally, we introduce an Iterative Cross-view Gaussians Alignment method to ensure consistent depth scales across different views. Our eFreeSplat represents an innovative approach for generalizable novel view synthesis. Different from the existing pure geometry-free methods, eFreeSplat focuses more on achieving epipolar-free feature matching and encoding by providing 3D priors through cross-view pretraining. We evaluate eFreeSplat on wide-baseline novel view synthesis tasks using the RealEstate10K and ACID datasets. Extensive experiments demonstrate that eFreeSplat surpasses state-of-the-art baselines that rely on epipolar priors, achieving superior geometry reconstruction and novel view synthesis quality. Project page: https://tatakai1.github.io/efreesplat/., Comment: NeurIPS 2024

Visual imitation learning (VIL) provides an efficient and intuitive strategy for robotic systems to acquire novel skills. Recent advancements in Vision Language Models (VLMs) have demonstrated remarkable performance in vision and language reasoning capabilities for VIL tasks. Despite the progress, current VIL methods naively employ VLMs to learn high-level plans from human videos, relying on pre-defined motion primitives for executing physical interactions, which remains a major bottleneck. In this work, we present VLMimic, a novel paradigm that harnesses VLMs to directly learn even fine-grained action levels, only given a limited number of human videos. Specifically, VLMimic first grounds object-centric movements from human videos, and learns skills using hierarchical constraint representations, facilitating the derivation of skills with fine-grained action levels from limited human videos. These skills are refined and updated through an iterative comparison strategy, enabling efficient adaptation to unseen environments. Our extensive experiments exhibit that our VLMimic, using only 5 human videos, yields significant improvements of over 27% and 21% in RLBench and real-world manipulation tasks, and surpasses baselines by over 37% in long-horizon tasks., Comment: accepted for publication in the 38th Conference on Neural Information Processing Systems (NeurIPS 2024)

Synchrotron X-ray emission has been detected from nearly a dozen young supernova remnants (SNRs). X-rays of synchrotron origin exhibit linear polarization in a regular, non-randomly oriented magnetic field. The significant polarized X-ray emission from four such SNRs has already been reported on the basis of observations with the Imaging X-ray Polarimetry Explorer (IXPE). The magnetic-field structure as derived from IXPE observations is radial for Cassiopeia A, Tycho's SNR, and SN 1006, and tangential for RX J1713.7-3946. The latter together with the recent detection of a tangential magnetic field in SNR 1E 0102.2-7219 by the Australia Telescope Compact Array in the radio band shows that tangential magnetic fields can also be present in young SNRs. Thus, the dichotomy in polarization between young and middle-aged SNRs (radial magnetic fields in young SNRs, but tangential magnetic fields in middle-aged SNRs), previously noticed in the radio band, deserves additional attention. The present analysis of IXPE observations determines, for the first time, a magnetic-field structure in the northwestern rim of Vela Jr, also known as RX J0852.0-4622, and provides a new example of a young SNR with a tangential magnetic field., Comment: Accepted for publication in Astronomy and Astrophysics

Prevalent human-object interaction (HOI) detection approaches typically leverage large-scale visual-linguistic models to help recognize events involving humans and objects. Though promising, models trained via contrastive learning on text-image pairs often neglect mid/low-level visual cues and struggle at compositional reasoning. In response, we introduce DIFFUSIONHOI, a new HOI detector shedding light on text-to-image diffusion models. Unlike the aforementioned models, diffusion models excel in discerning mid/low-level visual concepts as generative models, and possess strong compositionality to handle novel concepts expressed in text inputs. Considering diffusion models usually emphasize instance objects, we first devise an inversion-based strategy to learn the expression of relation patterns between humans and objects in embedding space. These learned relation embeddings then serve as textual prompts, to steer diffusion models generate images that depict specific interactions, and extract HOI-relevant cues from images without heavy fine-tuning. Benefited from above, DIFFUSIONHOI achieves SOTA performance on three datasets under both regular and zero-shot setups., Comment: NeurIPS 2024

Electron-beam microscopy and spectroscopy featuring atomic-scale spatial resolution have become essential tools used daily in almost all branches of nanoscale science and technology. As a natural supercontinuum source of light, free electrons couple with phonons, plasmons, electron-hole pairs, inter- and intra-band transitions, and inner-shell ionization. The multiple excitations, intertwined with the intricate nature of nanostructured samples, present significant challenges in isolating specific spectral characteristics amidst complex experimental backgrounds. Here we introduce the approach of synthetic complex frequency waves to mitigate these challenges in free-electron--light interaction. The complex frequency waves, created through causality-informed coherent superposition of real-frequency waves induced by free electrons, offer virtual gain to offset material losses. This amplifies and enhances spectral features, as confirmed by our electron energy loss and cathodoluminescence measurements on multi-layer membranes, suspended nanoparticles, and film-coupled nanostructures. Strikingly, we reveal that our approach can retrieve resonance excitation completed buried underneath the zero-loss peak, substantially enhance the quality of hyperspectral imaging, and resolve entangled multiple-photon--electron events in their quantum interaction. Our findings indicate the versatile utility of complex frequency waves in various electron-beam spectroscopy and their promising diagnostic capabilities in free-electron quantum optics.

Accumulated detailed knowledge about the neuronal activities in human brains has brought more attention to bio-inspired spiking neural networks (SNNs). In contrast to non-spiking deep neural networks (DNNs), SNNs can encode and transmit spatiotemporal information more efficiently by exploiting biologically realistic and low-power event-driven neuromorphic architectures. However, the supervised learning of SNNs still remains a challenge because the spike-timing-dependent plasticity (STDP) of connected spiking neurons is difficult to implement and interpret in existing backpropagation learning schemes. This paper proposes a fractional-order spike-timing-dependent gradient descent (FO-STDGD) learning model by considering a derived nonlinear activation function that describes the relationship between the quasi-instantaneous firing rate and the temporal membrane potentials of nonleaky integrate-and-fire neurons. The training strategy can be generalized to any fractional orders between 0 and 2 since the FO-STDGD incorporates the fractional gradient descent method into the calculation of spike-timing-dependent loss gradients. The proposed FO-STDGD model is tested on the MNIST and DVS128 Gesture datasets and its accuracy under different network structure and fractional orders is analyzed. It can be found that the classification accuracy increases as the fractional order increases, and specifically, the case of fractional order 1.9 improves by 155% relative to the case of fractional order 1 (traditional gradient descent). In addition, our scheme demonstrates the state-of-the-art computational efficacy for the same SNN structure and training epochs., Comment: 15 pages, 12 figures

Vision-language navigation (VLN) requires an agent to execute actions following human instructions. Existing VLN models are optimized through expert demonstrations by supervised behavioural cloning or incorporating manual reward engineering. While straightforward, these efforts overlook the accumulation of errors in the Markov decision process, and struggle to match the distribution of the expert policy. Going beyond this, we propose an Energy-based Navigation Policy (ENP) to model the joint state-action distribution using an energy-based model. At each step, low energy values correspond to the state-action pairs that the expert is most likely to perform, and vice versa. Theoretically, the optimization objective is equivalent to minimizing the forward divergence between the occupancy measure of the expert and ours. Consequently, ENP learns to globally align with the expert policy by maximizing the likelihood of the actions and modeling the dynamics of the navigation states in a collaborative manner. With a variety of VLN architectures, ENP achieves promising performances on R2R, REVERIE, RxR, and R2R-CE, unleashing the power of existing VLN models.

While multimodal large language models (MLLMs) have demonstrated extraordinary vision-language understanding capabilities and shown potential to serve as general-purpose assistants, their abilities to solve instance-level visual-language problems beyond a single image warrant further exploration. In order to assess these unproven abilities of MLLMs, this paper proposes a new visual grounding task called multi-context visual grounding, which aims to localize instances of interest across multiple images based on open-ended text prompts. To facilitate this research, we meticulously construct a new dataset MC-Bench for benchmarking the visual grounding capabilities of MLLMs. MC-Bench features 2K high-quality and manually annotated samples, consisting of instance-level labeled image pairs and corresponding text prompts that indicate the target instances in the images. In total, there are three distinct styles of text prompts, covering 20 practical skills. We benchmark over 20 state-of-the-art MLLMs and foundation models with potential multi-context visual grounding capabilities. Our evaluation reveals a non-trivial performance gap between existing MLLMs and humans across all metrics. We also observe that existing MLLMs typically outperform foundation models without LLMs only on image-level metrics, and the specialist MLLMs trained on single images often struggle to generalize to multi-image scenarios. Moreover, a simple stepwise baseline integrating advanced MLLM and a detector can significantly surpass prior end-to-end MLLMs. We hope our MC-Bench and empirical findings can encourage the research community to further explore and enhance the untapped potentials of MLLMs in instance-level tasks, particularly in multi-image contexts. Project page: https://xuyunqiu.github.io/MC-Bench/.

The increasing demand for controllable outputs in text-to-image generation has spurred advancements in multi-instance generation (MIG), allowing users to define both instance layouts and attributes. However, unlike image-conditional generation methods such as ControlNet, MIG techniques have not been widely adopted in state-of-the-art models like SD2 and SDXL, primarily due to the challenge of building robust renderers that simultaneously handle instance positioning and attribute rendering. In this paper, we introduce Depth-Driven Decoupled Instance Synthesis (3DIS), a novel framework that decouples the MIG process into two stages: (i) generating a coarse scene depth map for accurate instance positioning and scene composition, and (ii) rendering fine-grained attributes using pre-trained ControlNet on any foundational model, without additional training. Our 3DIS framework integrates a custom adapter into LDM3D for precise depth-based layouts and employs a finetuning-free method for enhanced instance-level attribute rendering. Extensive experiments on COCO-Position and COCO-MIG benchmarks demonstrate that 3DIS significantly outperforms existing methods in both layout precision and attribute rendering. Notably, 3DIS offers seamless compatibility with diverse foundational models, providing a robust, adaptable solution for advanced multi-instance generation. The code is available at: https://github.com/limuloo/3DIS., Comment: 10 pages

Two typical neural models have been extensively studied for operator learning, learning in spatial space via attention mechanism or learning in spectral space via spectral analysis technique such as Fourier Transform. Spatial learning enables point-level flexibility but lacks global continuity constraint, while spectral learning enforces spectral continuity prior but lacks point-wise adaptivity. This work innovatively combines the continuity prior and the point-level flexibility, with the introduced Point-Calibrated Spectral Transform. It achieves this by calibrating the preset spectral eigenfunctions with the predicted point-wise frequency preference via neural gate mechanism. Beyond this, we introduce Point-Calibrated Spectral Neural Operators, which learn operator mappings by approximating functions with the point-level adaptive spectral basis, thereby not only preserving the benefits of spectral prior but also boasting the superior adaptability comparable to the attention mechanism. Comprehensive experiments demonstrate its consistent performance enhancement in extensive PDE solving scenarios.

Dust from core-collapse supernovae (CCSNe), specifically Type IIP SNe, has been suggested to be a significant source of the dust observed in high-redshift galaxies. CCSNe eject large amounts of newly formed heavy elements, which can condense into dust grains in the cooling ejecta. However, infrared (IR) observations of typical CCSNe generally measure dust masses that are too small to account for the dust production needed at high redshifts. Type IIn SNe, classified by their dense circumstellar medium (CSM), are also known to exhibit strong IR emission from warm dust, but the dust origin and heating mechanism have generally remained unconstrained because of limited observational capabilities in the mid-IR. Here, we present a JWST/MIRI Medium Resolution Spectrograph (MRS) spectrum of the Type IIn SN 2005ip nearly 17 years post-explosion. The Type IIn SN 2005ip is one of the longest-lasting and most well-studied SNe observed to date. Combined with a Spitzer mid-IR spectrum of SN 2005ip obtained in 2008, this data set provides a rare 15-year baseline, allowing for a unique investigation of the evolution of dust. The JWST spectrum shows a new high-mass dust component ($\gtrsim0.08$ M$_{\odot}$) that is not present in the earlier Spitzer spectrum. Our analysis shows dust likely formed over the past 15 years in the cold, dense shell (CDS), between the forward and reverse shocks. There is also a smaller mass of carbonaceous dust ($\gtrsim0.005$ M$_{\odot}$) in the ejecta. These observations provide new insights into the role of SN dust production, particularly within the CDS, and its potential contribution to the rapid dust enrichment of the early Universe.

We perform the first Lattice calculation about the charmed hadron spin decomposition using overlap fermions on a 2+1 flavor RBC/UKQCD domain-wall gauge configurations at 0.083 fm with 300 MeV pion mass. It is found that the contributions of quark spin to the spin of 1S, 1P charmonia and also proton-like triple heavy quark state are comparable with the expectation of non-relativistic quark model. Such an observation provides evidence that the non-triviality of proton spin decomposition mainly arises from the relativistic effects of the light quark. Conversely, the substantial gluon angular momentum contribution in the spin $(1/2)^+$ state with triple heavy quarks at the charm quark mass, remains significant, highlighting the ongoing importance of the gluon in the realm of charmed baryon physics., Comment: 10 pages, 7 figures

Semantic segmentation is a critical technique for effective scene understanding. Traditional RGB-T semantic segmentation models often struggle to generalize across diverse scenarios due to their reliance on pretrained models and predefined categories. Recent advancements in Visual Language Models (VLMs) have facilitated a shift from closed-set to open-vocabulary semantic segmentation methods. However, these models face challenges in dealing with intricate scenes, primarily due to the heterogeneity between RGB and thermal modalities. To address this gap, we present Open-RGBT, a novel open-vocabulary RGB-T semantic segmentation model. Specifically, we obtain instance-level detection proposals by incorporating visual prompts to enhance category understanding. Additionally, we employ the CLIP model to assess image-text similarity, which helps correct semantic consistency and mitigates ambiguities in category identification. Empirical evaluations demonstrate that Open-RGBT achieves superior performance in diverse and challenging real-world scenarios, even in the wild, significantly advancing the field of RGB-T semantic segmentation.

Electric guitar tone modeling typically focuses on the non-linear transformation from clean to amplifier-rendered audio. Traditional methods rely on one-to-one mappings, incorporating device parameters into neural models to replicate specific amplifiers. However, these methods are limited by the need for specific training data. In this paper, we adapt a model based on the previous work, which leverages a tone embedding encoder and a feature wise linear modulation (FiLM) condition method. In this work, we altered conditioning method using a hypernetwork-based gated convolutional network (GCN) to generate audio that blends clean input with the tone characteristics of reference audio. By extending the training data to cover a wider variety of amplifier tones, our model is able to capture a broader range of tones. Additionally, we developed a real-time plugin to demonstrate the system's practical application, allowing users to experience its performance interactively. Our results indicate that the proposed system achieves superior tone modeling versatility compared to traditional methods., Comment: demo of the ISMIR paper

In recent years, 3D parametric animal models have been developed to aid in estimating 3D shape and pose from images and video. While progress has been made for humans, it's more challenging for animals due to limited annotated data. To address this, we introduce the first method using synthetic data generation and disentanglement to learn to regress 3D shape and pose. Focusing on horses, we use text-based texture generation and a synthetic data pipeline to create varied shapes, poses, and appearances, learning disentangled spaces. Our method, Dessie, surpasses existing 3D horse reconstruction methods and generalizes to other large animals like zebras, cows, and deer. See the project website at: \url{https://celiali.github.io/Dessie/}., Comment: ACCV2024

Lexical tones play a crucial role in Sino-Tibetan languages. However, current phonetic fieldwork relies on manual effort, resulting in substantial time and financial costs. This is especially challenging for the numerous endangered languages that are rapidly disappearing, often compounded by limited funding. In this paper, we introduce pitch-based similarity representations for tone transcription, named Tone2Vec. Experiments on dialect clustering and variance show that Tone2Vec effectively captures fine-grained tone variation. Utilizing Tone2Vec, we develop the first automatic approach for tone transcription and clustering by presenting a novel representation transformation for transcriptions. Additionally, these algorithms are systematically integrated into an open-sourced and easy-to-use package, ToneLab, which facilitates automated fieldwork and cross-regional, cross-lexical analysis for tonal languages. Extensive experiments were conducted to demonstrate the effectiveness of our methods., Comment: Accepted by EMNLP 2024 Findings

Automatic synthesis of analog circuits presents significant challenges. Existing methods usually treat the task as optimization problems, which limits their transferability and reusability for new requirements. To address this limitation, we introduce a task that directly generates analog circuits based on specified specifications, termed specification-conditioned analog circuit generation. Specifically, we propose CktGen, a simple yet effective variational autoencoder (VAE) model, that maps specifications and circuits into a joint latent space, and reconstructs the circuit from the latent. Moreover, given that a single specification can correspond to multiple distinct circuits, simply minimizing the distance between the mapped latent representations of the circuit and specification does not capture these one-to-many relationships. To address this, we integrate contrastive learning and classifier guidance to prevent model collapse. We conduct comprehensive experiments on the Open Circuit Benchmark (OCB) and introduce new evaluation metrics for cross-model consistency in the specification-to-circuit generation task. Experimental results demonstrate substantial improvements over existing state-of-the-art methods.

Images produced by diffusion models are increasingly popular in digital artwork and visual marketing. However, such generated images might replicate content from existing ones and pose the challenge of content originality. Existing Image Copy Detection (ICD) models, though accurate in detecting hand-crafted replicas, overlook the challenge from diffusion models. This motivates us to introduce ICDiff, the first ICD specialized for diffusion models. To this end, we construct a Diffusion-Replication (D-Rep) dataset and correspondingly propose a novel deep embedding method. D-Rep uses a state-of-the-art diffusion model (Stable Diffusion V1.5) to generate 40, 000 image-replica pairs, which are manually annotated into 6 replication levels ranging from 0 (no replication) to 5 (total replication). Our method, PDF-Embedding, transforms the replication level of each image-replica pair into a probability density function (PDF) as the supervision signal. The intuition is that the probability of neighboring replication levels should be continuous and smooth. Experimental results show that PDF-Embedding surpasses protocol-driven methods and non-PDF choices on the D-Rep test set. Moreover, by utilizing PDF-Embedding, we find that the replication ratios of well-known diffusion models against an open-source gallery range from 10% to 20%. The project is publicly available at https://icdiff.github.io/., Comment: Accepted by NeurIPS 2024

Recent advancements in quantum polarization theory have propelled the exploration of topological insulators (TIs) into the realm of higher-order systems, leading to the study of the celebrated two-dimensional (2D) quadrupole and three-dimensional (3D) octupole TIs. Traditionally, these topological phases have been associated with the toroidal topology of the conventional Brillouin zone (BZ). This Letter reports on the discovery of a novel octupole topological insulating phase emerging within the framework of the Brillouin 3D real projective space ($\mathbb{RP}^3$). We theoretically propose the model and its corresponding topological invariant, experimentally construct this insulator within a topological circuit framework, and capture the octupole insulating phase as a localized impedance peak at the circuit's corner. Furthermore, our $\mathbb{RP}^3$ circuit stands out as a pioneering 3D model to simultaneously exhibit both intrinsic, termination-independent symmetry-protected topological phases (SPTPs) and extrinsic, termination-dependent boundary-obstructed topological phases (BOTPs), which broadly encompass 2D surface-obstructed topological phases (SOTPs) and 1D hinge-obstructed topological phases (HOTPs). Our results broaden the topological landscape and provide insights into the band theory within the manifold of the Brillouin $\mathbb{RP}^3$., Comment: 6 pages, 4 figures

Embedding models play a crucial role in representing and retrieving information across various NLP applications. Recent advancements in Large Language Models (LLMs) have further enhanced the performance of embedding models, which are trained on massive amounts of text covering almost every domain. These models are often benchmarked on general-purpose datasets like Massive Text Embedding Benchmark (MTEB), where they demonstrate superior performance. However, a critical question arises: Is the development of domain-specific embedding models necessary when general-purpose models are trained on vast corpora that already include specialized domain texts? In this paper, we empirically investigate this question, choosing the finance domain as an example. We introduce the Finance Massive Text Embedding Benchmark (FinMTEB), a counterpart to MTEB that consists of financial domain-specific text datasets. We evaluate the performance of seven state-of-the-art embedding models on FinMTEB and observe a significant performance drop compared to their performance on MTEB. To account for the possibility that this drop is driven by FinMTEB's higher complexity, we propose four measures to quantify dataset complexity and control for this factor in our analysis. Our analysis provides compelling evidence that state-of-the-art embedding models struggle to capture domain-specific linguistic and semantic patterns. Moreover, we find that the performance of general-purpose embedding models on MTEB is not correlated with their performance on FinMTEB, indicating the need for domain-specific embedding benchmarks for domain-specific embedding models. This study sheds light on developing domain-specific embedding models in the LLM era. FinMTEB comes with open-source code at https://github.com/yixuantt/FinMTEB, Comment: https://github.com/yixuantt/FinMTEB

The Feibelman $d$-parameter, a mesoscopic complement to the local bulk permittivity, describes quantum optical surface responses for interfaces, including nonlocality, spill-in and-out, and surface-enabled Landau damping. It has been incorporated into the macroscopic Maxwellian framework for convenient modeling and understanding of nanoscale electromagnetic phenomena, calling for the compilation of a $d$-parameter database for interfaces of interest in nano-optics. However, accurate first-principles calculations of $d$-parameters face computational challenges, whereas existing measurements of $d$-parameters are scarce and restricted to narrow spectral windows. We demonstrate a general broadband ellipsometric approach to measure $d$-parameters at a gold--air interface across the visible--ultraviolet regimes. Gold is found to spill in and spill out at different frequencies. We also observe gold's Bennett mode, a surface-dipole resonance associated with a pole of the $d$-parameter, around 2.5 eV. Our measurements give rise to and are further validated by the passivity and Kramers--Kronig causality analysis of $d$-parameters. Our work advances the understanding of quantum surface response and may enable applications like enhanced electron field emission.

The increasing use of text as data in social science research necessitates the development of valid, consistent, reproducible, and efficient methods for generating text-based concept measures. This paper presents a novel method that leverages the internal hidden states of large language models (LLMs) to generate these concept measures. Specifically, the proposed method learns a concept vector that captures how the LLM internally represents the target concept, then estimates the concept value for text data by projecting the text's LLM hidden states onto the concept vector. Three replication studies demonstrate the method's effectiveness in producing highly valid, consistent, and reproducible text-based measures across various social science research contexts, highlighting its potential as a valuable tool for the research community.

We introduce a novel Recursive Fusion model, dubbed ReFu, designed to integrate point clouds and meshes for exemplar-free 3D Class-Incremental Learning, where the model learns new 3D classes while retaining knowledge of previously learned ones. Unlike existing methods that either rely on storing historical data to mitigate forgetting or focus on single data modalities, ReFu eliminates the need for exemplar storage while utilizing the complementary strengths of both point clouds and meshes. To achieve this, we introduce a recursive method which continuously accumulates knowledge by updating the regularized auto-correlation matrix. Furthermore, we propose a fusion module, featuring a Pointcloud-guided Mesh Attention Layer that learns correlations between the two modalities. This mechanism effectively integrates point cloud and mesh features, leading to more robust and stable continual learning. Experiments across various datasets demonstrate that our proposed framework outperforms existing methods in 3D class-incremental learning. Project Page: https://arlo397.github.io/ReFu/

Western music is often characterized by a homophonic texture, in which the musical content can be organized into a melody and an accompaniment. In orchestral music, in particular, the composer can select specific characteristics for each instrument's part within the accompaniment, while also needing to adapt the melody to suit the capabilities of the instruments performing it. In this work, we propose METEOR, a model for Melody-aware Texture-controllable Orchestral music generation. This model performs symbolic multi-track music style transfer with a focus on melodic fidelity. We allow bar- and track-level controllability of the accompaniment with various textural attributes while keeping a homophonic texture. We show that the model can achieve controllability performances similar to strong baselines while greatly improve melodic fidelity., Comment: https://dinhviettoanle.github.io/meteor

DETR introduces a simplified one-stage framework for scene graph generation (SGG). However, DETR-based SGG models face two challenges: i) Sparse supervision, as each image typically contains fewer than 10 relation annotations, while the models employ over 100 relation queries. This sparsity arises because each ground truth relation is assigned to only one single query during training. ii) False negative samples, since one ground truth relation may have multiple queries with similar matching scores. These suboptimally matched queries are simply treated as negative samples, causing the loss of valuable supervisory signals. As a response, we devise Hydra-SGG, a one-stage SGG method that adopts a new Hybrid Relation Assignment. This assignment combines a One-to-One Relation Assignment with a newly introduced IoU-based One-to-Many Relation Assignment. Specifically, each ground truth is assigned to multiple relation queries with high IoU subject-object boxes. This Hybrid Relation Assignment increases the number of positive training samples, alleviating sparse supervision. Moreover, we, for the first time, empirically show that self-attention over relation queries helps reduce duplicated relation predictions. We, therefore, propose Hydra Branch, a parameter-sharing auxiliary decoder without a self-attention layer. This design promotes One-to-Many Relation Assignment by enabling different queries to predict the same relation. Hydra-SGG achieves state-of-the-art performance with 10.6 mR@20 and 16.0 mR@50 on VG150, while only requiring 12 training epochs. It also sets a new state-of-the-art on Open Images V6 and and GQA.

In this paper, we propose an LLM-empowered RM-API misuse detection solution, ChatDetector, which fully automates LLMs for documentation understanding which helps RM-API constraints retrieval and RM-API misuse detection. To correctly retrieve the RM-API constraints, ChatDetector is inspired by the ReAct framework which is optimized based on Chain-of-Thought (CoT) to decompose the complex task into allocation APIs identification, RM-object (allocated/released by RM APIs) extraction and RM-APIs pairing (RM APIs usually exist in pairs). It first verifies the semantics of allocation APIs based on the retrieved RM sentences from API documentation through LLMs. Inspired by the LLMs' performance on various prompting methods,ChatDetector adopts a two-dimensional prompting approach for cross-validation. At the same time, an inconsistency-checking approach between the LLMs' output and the reasoning process is adopted for the allocation APIs confirmation with an off-the-shelf Natural Language Processing (NLP) tool. To accurately pair the RM-APIs, ChatDetector decomposes the task again and identifies the RM-object type first, with which it can then accurately pair the releasing APIs and further construct the RM-API constraints for misuse detection. With the diminished hallucinations, ChatDetector identifies 165 pairs of RM-APIs with a precision of 98.21% compared with the state-of-the-art API detectors. By employing a static detector CodeQL, we ethically report 115 security bugs on the applications integrating on six popular libraries to the developers, which may result in severe issues, such as Denial-of-Services (DoS) and memory corruption. Compared with the end-to-end benchmark method, the result shows that ChatDetector can retrieve at least 47% more RM sentences and 80.85% more RM-API constraints., Comment: Accepted by NDSS Symposium 2025. Please cite this paper as "Yi Yang, Jinghua Liu, Kai Chen, Miaoqian Lin. The Midas Touch: Triggering the Capability of LLMs for RM-API Misuse Detection. In the 32nd Annual Network and Distributed System Security Symposium (NDSS 2025)."

In this paper, we present a new framework, named GPTAid, for automatic APSRs generation by analyzing API source code with LLM and detecting API misuse caused by incorrect parameter use. To validate the correctness of the LLM-generated APSRs, we propose an execution feedback-checking approach based on the observation that security-critical API misuse is often caused by APSRs violations, and most of them result in runtime errors. Specifically, GPTAid first uses LLM to generate raw APSRs and the Right calling code, and then generates Violation code for each raw APSR by modifying the Right calling code using LLM. Subsequently, GPTAid performs dynamic execution on each piece of Violation code and further filters out the incorrect APSRs based on runtime errors. To further generate concrete APSRs, GPTAid employs a code differential analysis to refine the filtered ones. Particularly, as the programming language is more precise than natural language, GPTAid identifies the key operations within Violation code by differential analysis, and then generates the corresponding concrete APSR based on the aforementioned operations. These concrete APSRs could be precisely interpreted into applicable detection code, which proven to be effective in API misuse detection. Implementing on the dataset containing 200 randomly selected APIs from eight popular libraries, GPTAid achieves a precision of 92.3%. Moreover, it generates 6 times more APSRs than state-of-the-art detectors on a comparison dataset of previously reported bugs and APSRs. We further evaluated GPTAid on 47 applications, 210 unknown security bugs were found potentially resulting in severe security issues (e.g., system crashes), 150 of which have been confirmed by developers after our reports., Comment: Accepted by NDSS Symposium 2025. Please cite this paper as "Jinghua Liu, Yi Yang, Kai Chen, and Miaoqian Lin. Generating API Parameter Security Rules with LLM for API Misuse Detection. In the 32nd Annual Network and Distributed System Security Symposium (NDSS 2025)

While deep neural networks have made remarkable progress in various vision tasks, their performance typically deteriorates when tested in out-of-distribution (OOD) scenarios. Many OOD methods focus on extracting domain-invariant features but neglect whether these features are unique to each class. Even if some features are domain-invariant, they cannot serve as key classification criteria if shared across different classes. In OOD tasks, both domain-related and class-shared features act as confounders that hinder generalization. In this paper, we propose a DICS model to extract Domain-Invariant and Class-Specific features, including Domain Invariance Testing (DIT) and Class Specificity Testing (CST), which mitigate the effects of spurious correlations introduced by confounders. DIT learns domain-related features of each source domain and removes them from inputs to isolate domain-invariant class-related features. DIT ensures domain invariance by aligning same-class features across different domains. Then, CST calculates soft labels for those features by comparing them with features learned in previous steps. We optimize the cross-entropy between the soft labels and their true labels, which enhances same-class similarity and different-class distinctiveness, thereby reinforcing class specificity. Extensive experiments on widely-used benchmarks demonstrate the effectiveness of our proposed algorithm. Additional visualizations further demonstrate that DICS effectively identifies the key features of each class in target domains.

Momentum-space nonsymmorphic symmetries, stemming from the projective algebra of synthetic gauge fields, can modify the manifold of the Brillouin zone and lead to a variety of topological phenomena. We present an acoustic realization of higher-order topological insulators (HOTIs) protected by a pair of anticommutative momentum-space glide reflections. We confirm the presence of momentum-space glide reflection from the measured momentum half translation of edge bands and their momentum-resolved probability distribution using a cylinder geometry made of acoustic resonator arrays. In particular, we observe both intrinsic and extrinsic HOTI features in such a cylinder: hopping strength variation along the open boundary leads to a bulk gap closure, while that along the closed boundary results in an edge gap closure. In addition, we confirm the presence of quadrupole corner modes with transmission and field distribution measurements. Our observation enriches the study of topological physics of momentum-space nonsymmorphic symmetries., Comment: 5 pages, 4 figures

Recent advancements in state-of-the-art (SOTA) Large Language Model (LLM) agents, especially in multi-turn dialogue tasks, have been primarily driven by supervised fine-tuning and high-quality human feedback. However, as base LLM models continue to improve, acquiring meaningful human feedback has become increasingly challenging and costly. In certain domains, base LLM agents may eventually exceed human capabilities, making traditional feedback-driven methods impractical. In this paper, we introduce a novel self-improvement paradigm that empowers LLM agents to autonomously enhance their performance without external human feedback. Our method, Juxtaposed Outcomes for Simulation Harvesting (JOSH), is a self-alignment algorithm that leverages a sparse reward simulation environment to extract ideal behaviors and further train the LLM on its own outputs. We present ToolWOZ, a sparse reward tool-calling simulation environment derived from MultiWOZ. We demonstrate that models trained with JOSH, both small and frontier, significantly improve tool-based interactions while preserving general model capabilities across diverse benchmarks. Our code and data are publicly available on GitHub at https://github.com/asappresearch/josh-llm-simulation-training, Comment: Minor but nontrivial changes likely

The significant advancements of Large Language Models (LLMs) in generative tasks have led to a growing body of work exploring LLM-based embedding models. While these models, employing different pooling and attention strategies, have achieved state-of-the-art performance on public embedding benchmarks, questions still arise about what constitutes an effective design for LLM-based embedding models. However, these models are often trained on different datasets, using different LLM base models or training settings. Moreover, evaluations on public embedding benchmarks often fail to report statistical significance, making it difficult to determine which designs truly contribute to final performance. This complicates the process for practitioners seeking optimal training recipes for LLM-based embedding models. In this study, we conduct a large-scale experiment by training a series of LLM-based embedding models using the same training data and base model but differing in their pooling and attention strategies. The results show that there is no one-size-fits-all solution: while bidirectional attention and an additional trainable pooling layer outperform in text similarity and information retrieval tasks, they do not significantly surpass simpler designs like EOS-last token pooling and default causal attention in clustering and classification tasks. Furthermore, we propose a new pooling strategy, Multi-Layers Trainable Pooling, which transforms the outputs of all hidden layers, rather than just the last layer, using a cross-attention network. This method proves to be statistically superior in text similarity and retrieval tasks compared to existing pooling methods. Overall, this paper sheds light on effective training strategies for LLM-based embedding models., Comment: https://github.com/yixuantt/PoolingAndAttn

The cores of stars are the cosmic furnaces where light elements are fused into heavier nuclei. The fusion of hydrogen to helium initially powers all stars. The ashes of the fusion reactions are then predicted to serve as fuel in a series of stages, eventually transforming massive stars into a structure of concentric shells. These are composed of natal hydrogen on the outside, and consecutively heavier compositions inside, predicted to be dominated by helium, carbon/oxygen, oxygen/neon/magnesium, and oxygen/silicon/sulphur. Silicon and sulphur are fused into inert iron, leading to the collapse of the core and either a supernova explosion or the direct formation of a black hole. Stripped stars, where the outer hydrogen layer has been removed and the internal He-rich layer (in Wolf-Rayet WN stars) or even the C/O layer below it (in Wolf-Rayet WC/WO stars) are exposed, provide evidence for this shell structure, and the cosmic element production mechanism it reflects. The types of supernova explosions that arise from stripped stars embedded in shells of circumstellar material (most notably Type Ibn supernovae from stars with outer He layers, and Type Icn supernovae from stars with outer C/O layers) confirm this scenario. However, direct evidence for the most interior shells, which are responsible for the production of elements heavier than oxygen, is lacking. Here, we report the discovery of the first-of-its-kind supernova arising from a star peculiarly stripped all the way to the silicon and sulphur-rich internal layer. Whereas the concentric shell structure of massive stars is not under debate, it is the first time that such a thick, massive silicon and sulphur-rich shell, expelled by the progenitor shortly before the SN explosion, has been directly revealed., Comment: 48 pages, 12 figures and 10 tables. Submitted to a high-impact journal. The reduced spectra and photometry will be made available via the journal webpage and the WISeREP archive after the acceptance of the paper

Cervical disc herniation (CDH) is a prevalent musculoskeletal disorder that significantly impacts health and requires labor-intensive analysis from experts. Despite advancements in automated detection of medical imaging, two significant challenges hinder the real-world application of these methods. First, the computational complexity and resource demands present a significant gap for real-time application. Second, noise in MRI reduces the effectiveness of existing methods by distorting feature extraction. To address these challenges, we propose three key contributions: Firstly, we introduced MedDet, which leverages the multi-teacher single-student knowledge distillation for model compression and efficiency, meanwhile integrating generative adversarial training to enhance performance. Additionally, we customize the second-order nmODE to improve the model's resistance to noise in MRI. Lastly, we conducted comprehensive experiments on the CDH-1848 dataset, achieving up to a 5% improvement in mAP compared to previous methods. Our approach also delivers over 5 times faster inference speed, with approximately 67.8% reduction in parameters and 36.9% reduction in FLOPs compared to the teacher model. These advancements significantly enhance the performance and efficiency of automated CDH detection, demonstrating promising potential for future application in clinical practice. See project website https://steve-zeyu-zhang.github.io/MedDet, Comment: Accepted to BIBM 2024 Oral

Zero-shot learning (ZSL) aims to recognize unseen classes by transferring semantic knowledge from seen classes to unseen ones, guided by semantic information. To this end, existing works have demonstrated remarkable performance by utilizing global visual features from Convolutional Neural Networks (CNNs) or Vision Transformers (ViTs) for visual-semantic interactions. Due to the limited receptive fields of CNNs and the quadratic complexity of ViTs, however, these visual backbones achieve suboptimal visual-semantic interactions. In this paper, motivated by the visual state space model (i.e., Vision Mamba), which is capable of capturing long-range dependencies and modeling complex visual dynamics, we propose a parameter-efficient ZSL framework called ZeroMamba to advance ZSL. Our ZeroMamba comprises three key components: Semantic-aware Local Projection (SLP), Global Representation Learning (GRL), and Semantic Fusion (SeF). Specifically, SLP integrates semantic embeddings to map visual features to local semantic-related representations, while GRL encourages the model to learn global semantic representations. SeF combines these two semantic representations to enhance the discriminability of semantic features. We incorporate these designs into Vision Mamba, forming an end-to-end ZSL framework. As a result, the learned semantic representations are better suited for classification. Through extensive experiments on four prominent ZSL benchmarks, ZeroMamba demonstrates superior performance, significantly outperforming the state-of-the-art (i.e., CNN-based and ViT-based) methods under both conventional ZSL (CZSL) and generalized ZSL (GZSL) settings. Code is available at: https://anonymous.4open.science/r/ZeroMamba.

The recently-discovered high-temperature superconductor La$_3$Ni$_2$O$_7$ under high pressure has stimulated intensive debates. Key controversies concern interlayer versus intralayer pairing scenarios and if the hybridization plays a key role in establishing the superconductivity. But experimental clarification is difficult due to the limitation of employing state-of-the-art techniques under high pressure. Here we propose that quasiparticle tunneling and Andreev reflection on a normal metal-superconductor junction may provide a feasible way to distinguish different pairing scenarios. We predict that an asymmetric Fano line shape may be induced by the hybridization between the $d_{x^2-y^2}$ metallic bands and the strongly renormalized flat $d_{z^2}$ quasiparticle bands. In the superconducting state, we show that the Andreev reflection should be greatly suppressed for interlayer pairing superconductivity with a small interlayer hopping. We propose future experiments to examine these predictions and help clarify the fundamental physics of superconducting La$_3$Ni$_2$O$_7$., Comment: 6 pages, 4 figures

We report a lattice QCD calculation of the parton distribution function (PDF) of a deuteron-like dibaryon system using large-momentum effective theory. The calculation is done on three Wilson Clover ensembles with a fixed lattice spacing a=0.105 fm and two pion masses. The lattice matrix elements are computed at proton momenta up to 2.46 GeV with the signal of high momentum modes being improved by applying the momentum smearing technique. The state-of-the-art renormalization, matching and extrapolation are then applied to obtain the final result of the light-cone PDF. A comparison between the result of the dibaryon system and the sum of the proton and neutron PDFs is also given.

Neural network models for guitar amplifier emulation, while being effective, often demand high computational cost and lack interpretability. Drawing ideas from physical amplifier design, this paper aims to address these issues with a new differentiable digital signal processing (DDSP)-based model, called ``DDSP guitar amp,'' that models the four components of a guitar amp (i.e., preamp, tone stack, power amp, and output transformer) using specific DSP-inspired designs. With a set of time- and frequency-domain metrics, we demonstrate that DDSP guitar amp achieves performance comparable with that of black-box baselines while requiring less than 10\% of the computational operations per audio sample, thereby holding greater potential for usages in real-time applications., Comment: Preprint paper

The Tsinghua University--Ma Huateng Telescopes for Survey (TMTS) started to monitor the LAMOST plates in 2020, leading to the discovery of numerous short-period eclipsing binaries, peculiar pulsators, flare stars, and other variable objects. Here, we present the uninterrupted light curves for a sample of 64 cataclysmic variables (CVs) observed/discovered using the TMTS during its first three-year observations, and we introduce new CVs and new light-variation periods (from known CVs) revealed through the TMTS observations. Thanks to the high-cadence observations of TMTS, diverse light variations, including superhumps, quasi-periodic oscillations, large-amplitude orbital modulations, and rotational modulations, are able to be detected in our CV samples, providing key observational clues for understanding the fast-developing physical processes in various CVs. All of these short-timescale light-curve features help further classify the subtypes of CV systems. We highlight the light-curve features observed in our CV sample and discuss further implications of minute-cadence light curves for CV identifications and classifications. Moreover, we examine the H$\alpha$ emission lines in the spectra from our nonmagnetic CV samples (i.e., dwarf novae and nova-like subclasses) and find that the distribution of H$\alpha$ emission strength shows significant differences between the sources with orbital periods above and below the period gap, which agrees with the trend seen from the SDSS nonmagnetic CV sample., Comment: 27 pages, 12 figures in main text, accepted for the publication in Universe

The recent discovery of high-temperature superconductivity in La$_3$Ni$_2$O$_7$ under high pressure has stimulated intensive investigations concerning its paring mechanism and a correct description of its effective low-energy physics. Notable experimental observations include the right-triangle-shaped superconducting dome with the maximum $T_c$ of about 80 K and the strange metal behavior in the normal state above $T_c$. Here we apply the Schwinger boson approach to the bilayer $t$-$V$-$J$ model, which allows us to treat well both the magnetic correlations and the superconducting instability. We obtain a global phase diagram with both metallic and superconducting ground states separated by a quantum phase transition, and predict a minimal interlayer superexchange $J$ and a minimal $d_{z^2}$-hole concentration necessary for the superconductivity, with a highly asymmetric right-triangle-like shape for the $T_c$ curve as well as a maximum $T_c$ comparable with experiments. The normal state is featured with a pseudogap in the $d_{x^2-y^2}$ spectra associated with preformed Cooper pairs and non-Fermi liquid strange metal behavior due to hybridization-induced spinon-holon-electron scattering. Our work clarifies the key difference of the two-component scenario of the superconductivity from other mechanisms, and provides a consistent understanding of the high-temperature superconductivity and strange metallic properties of La$_{3}$Ni$_2$O$_7$ under high pressure., Comment: 9 pages, 5 figures

Recent advancements in pre-trained Vision-Language Models (VLMs) have highlighted the significant potential of prompt tuning for adapting these models to a wide range of downstream tasks. However, existing prompt tuning methods typically map an image to a single representation, limiting the model's ability to capture the diverse ways an image can be described. To address this limitation, we investigate the impact of visual prompts on the model's generalization capability and introduce a novel method termed Multi-Representation Guided Prompt Tuning (MePT). Specifically, MePT employs a three-branch framework that focuses on diverse salient regions, uncovering the inherent knowledge within images which is crucial for robust generalization. Further, we employ efficient self-ensemble techniques to integrate these versatile image representations, allowing MePT to learn all conditional, marginal, and fine-grained distributions effectively. We validate the effectiveness of MePT through extensive experiments, demonstrating significant improvements on both base-to-novel class prediction and domain generalization tasks.

In this paper, modified Toda (mToda) equation is generalized to form an integrable hierarchy in the framework of Sato theory, which is therefore called mToda hierarchy. Inspired by the fact that Toda hierarchy is 2--component generalization of usual KP hierarchy, mToda hierarchy is constructed from bilinear equations of 2--component 1st modified KP hierarchy, where we provide the corresponding equivalence with Lax formulations. Then it is demonstrated that there are Miura links between Toda and mToda hierarchies, which means the definition of mToda hierarchy here is reasonable. Finally, Darboux transformations of the Toda and mToda hierarchies are also constructed by using the aforementioned Miura links., Comment: 23 pages

Robust surface states are a defining feature of topological photonic and acoustic systems. Although the supercell technique is commonly used for studying their surface effects, it falls short in sorting the states at two boundaries, differentiating surface states from bulk states, and demanding large computational resources. To overcome the limitations, we employ surface Green's functions combined with two efficient algorithms to obtain the ideal surface states for a semi-infinite system. They are further extended to more complex scenarios, such as coated structures, heterostructures, and sandwiched structures. Photonic Chern insulators, valley photonic crystals, and acoustic topological insulators are taken as examples for demonstration and verification. The surface spectra obtained from our approach can be directly used for comparison with experimental observables. Our approach enriches the efficient computational tools for topological photonics and acoustics., Comment: 11 pages, 7 figures

This paper focuses on Human-Object Interaction (HOI) detection, addressing the challenge of identifying and understanding the interactions between humans and objects within a given image or video frame. Spearheaded by Detection Transformer (DETR), recent developments lead to significant improvements by replacing traditional region proposals by a set of learnable queries. However, despite the powerful representation capabilities provided by Transformers, existing Human-Object Interaction (HOI) detection methods still yield low confidence levels when dealing with complex interactions and are prone to overlooking interactive actions. To address these issues, we propose a novel approach \textsc{UAHOI}, Uncertainty-aware Robust Human-Object Interaction Learning that explicitly estimates prediction uncertainty during the training process to refine both detection and interaction predictions. Our model not only predicts the HOI triplets but also quantifies the uncertainty of these predictions. Specifically, we model this uncertainty through the variance of predictions and incorporate it into the optimization objective, allowing the model to adaptively adjust its confidence threshold based on prediction variance. This integration helps in mitigating the adverse effects of incorrect or ambiguous predictions that are common in traditional methods without any hand-designed components, serving as an automatic confidence threshold. Our method is flexible to existing HOI detection methods and demonstrates improved accuracy. We evaluate \textsc{UAHOI} on two standard benchmarks in the field: V-COCO and HICO-DET, which represent challenging scenarios for HOI detection. Through extensive experiments, we demonstrate that \textsc{UAHOI} achieves significant improvements over existing state-of-the-art methods, enhancing both the accuracy and robustness of HOI detection., Comment: Accepted by CVIU

Starting from a fairly explicit homogeneous realization of the toroidal Lie algebra $\mathcal{L}^{\rm tor}_{r+1}(\mathfrak{sl}_\ell)$ via lattice vertex algebra, we derive an integrable hierarchy of Hirota bilinear equations. Moreover, we represent this hierarchy in the form of Lax equations, and show that it is an extension of a certain reduction of the $\ell$-component KP hierarchy.

We present PyNeuralFx, an open-source Python toolkit designed for research on neural audio effect modeling. The toolkit provides an intuitive framework and offers a comprehensive suite of features, including standardized implementation of well-established model architectures, loss functions, and easy-to-use visualization tools. As such, it helps promote reproducibility for research on neural audio effect modeling, and enable in-depth performance comparison of different models, offering insight into the behavior and operational characteristics of models through DSP methodology. The toolkit can be found at https://github.com/ytsrt66589/pyneuralfx., Comment: toolkit paper

We discover a general stringy scaling behavior for 1. All n-point hard string scattering amplitudes (HSSA) and 2. A class of n-point Regge string scattering amplitudes (RSSA) to all string loop orders. The number of independent kinematics variables is found to be reduced by dim M., Comment: 11 pages, 1 figure. Talk given by Jen-Chi Lee at Komaba, Tokyo university, Jan.29/2024

Recurrent neural networks (RNNs) have demonstrated impressive results for virtual analog modeling of audio effects. These networks process time-domain audio signals using a series of matrix multiplication and nonlinear activation functions to emulate the behavior of the target device accurately. To additionally model the effect of the knobs for an RNN-based model, existing approaches integrate control parameters by concatenating them channel-wisely with some intermediate representation of the input signal. While this method is parameter-efficient, there is room to further improve the quality of generated audio because the concatenation-based conditioning method has limited capacity in modulating signals. In this paper, we propose three novel conditioning mechanisms for RNNs, tailored for black-box virtual analog modeling. These advanced conditioning mechanisms modulate the model based on control parameters, yielding superior results to existing RNN- and CNN-based architectures across various evaluation metrics., Comment: Accepted to DAFx24

We present the determination of the charm quark mass, the masses and decay constants of charmed mesons using thirteen 2+1 flavor full-QCD gauge ensembles at five different lattice spacings $a\in[0.05,0.11]$ fm, 8 pion masses $m_{\pi}\in(130,360)$ MeV, and several values of the strange quark mass, which facilitate us to do the chiral and continuum extrapolation. These ensembles are generated through the stout smeared clover fermion action and Symanzik gauge actions with the tadpole improvement. Using QED-subtracted $D_s$ meson mass and non-perturbative renormalization, we predict the charm quark mass in the continuum with physical light and strange quark masses to be {$m_c(m_c)=1.289(17)$} GeV in $\overline{\textrm{MS}}$ scheme, with the systematic uncertainties from lattice spacing determination, renormalization constant, {and fit ansatz} included. Predictions of the open and close charm mesons using this charm quark mass agree with the experimental value at 0.3\% level uncertainty. We obtained {$D_{(s)}$ decay constants and also by far the most precise $D_{(s)}^*$ decay constants $f_{D^*}=0.2321(43)$ GeV and $f_{D^*_s}=0.2743(34)$ GeV}, with the charm quark improved vector current normalization., Comment: 14 pages, 14 figures

The extraction of Metal-Organic Frameworks (MOFs) synthesis conditions from literature text has been challenging but crucial for the logical design of new MOFs with desirable functionality. The recent advent of large language models (LLMs) provides disruptively new solution to this long-standing problem and latest researches have reported over 90% F1 in extracting correct conditions from MOFs literature. We argue in this paper that most existing synthesis extraction practices with LLMs stay with the primitive zero-shot learning, which could lead to downgraded extraction and application performance due to the lack of specialized knowledge. This work pioneers and optimizes the few-shot in-context learning paradigm for LLM extraction of material synthesis conditions. First, we propose a human-AI joint data curation process to secure high-quality ground-truth demonstrations for few-shot learning. Second, we apply a BM25 algorithm based on the retrieval-augmented generation (RAG) technique to adaptively select few-shot demonstrations for each MOF's extraction. Over a dataset randomly sampled from 84,898 well-defined MOFs, the proposed few-shot method achieves much higher average F1 performance (0.93 vs. 0.81, +14.8%) than the native zero-shot LLM using the same GPT-4 model, under fully automatic evaluation that are more objective than the previous human evaluation. The proposed method is further validated through real-world material experiments: compared with the baseline zero-shot LLM, the proposed few-shot approach increases the MOFs structural inference performance (R^2) by 29.4% in average.

In this paper, we briefly introduce the solution developed by our team, HFUT-VUT, for the track of Micro-gesture Classification in the MiGA challenge at IJCAI 2024. The task of micro-gesture classification task involves recognizing the category of a given video clip, which focuses on more fine-grained and subtle body movements compared to typical action recognition tasks. Given the inherent complexity of micro-gesture recognition, which includes large intra-class variability and minimal inter-class differences, we utilize two innovative modules, i.e., the cross-modal fusion module and prototypical refinement module, to improve the discriminative ability of MG features, thereby improving the classification accuracy. Our solution achieved significant success, ranking 1st in the track of Micro-gesture Classification. We surpassed the performance of last year's leading team by a substantial margin, improving Top-1 accuracy by 6.13%., Comment: 1st Place in Micro-gesture Classification in MiGA at IJCAI-2024