Search

The spatial-temporal evolution of coronal plasma parameters of the solar outer atmosphere at global scales, derived from solar full-disk imaging spectroscopic observation in the extreme-ultraviolet band, is critical for understanding and forecasting solar eruptions. We propose a multi-slits extreme ultraviolet imaging spectrograph for global coronal diagnostics with high cadence and present the preliminary instrument designs for the wavelength range from 18.3 to 19.8 nm. The instrument takes a comprehensive approach to obtain global coronal spatial and spectral information, improve the detected cadence and avoid overlapping. We first describe the relationship between optical properties and structural parameters, especially the relationship between the overlapping and the number of slits, and give a general multi-slits extreme-ultraviolet imaging spectrograph design process. Themultilayer structure is optimized to enhance the effective areas in the observation band. Five distantly-separated slits are set to divide the entire solar field of view, which increase the cadence for raster scanning the solar disk by 5 times relative to a single slit. The spectral resolving power of the optical system with an aperture diameter of 150 mm are optimized to be greater than 1461. The spatial resolution along the slits direction and the scanning direction are about 4.4''and 6.86'', respectively. The Al/Mo/B4C multilayer structure is optimized and the peak effective area is about 1.60 cm2 at 19.3 nm with a full width at half maximum of about 1.3 nm. The cadence to finish full-disk raster scan is about 5 minutes. Finally, the instrument performance is evaluated by an end-to-end calculation of the system photon budget and a simulation of the observational image and spectra. Our investigation shows that this approach is promising for global coronal plasma diagnostics., Comment: This version of the article has been accepted for publication, after peer review (when applicable) but is not the Version of Record and does not reflect post-acceptance improvements, or any corrections. The Version of Record is available online at: http://dx.doi.org/10.1007/s10686-024-09961-9

Plane instance segmentation from RGB-D data is a crucial research topic for many downstream tasks. However, most existing deep-learning-based methods utilize only information within the RGB bands, neglecting the important role of the depth band in plane instance segmentation. Based on EfficientSAM, a fast version of SAM, we propose a plane instance segmentation network called PlaneSAM, which can fully integrate the information of the RGB bands (spectral bands) and the D band (geometric band), thereby improving the effectiveness of plane instance segmentation in a multimodal manner. Specifically, we use a dual-complexity backbone, with primarily the simpler branch learning D-band features and primarily the more complex branch learning RGB-band features. Consequently, the backbone can effectively learn D-band feature representations even when D-band training data is limited in scale, retain the powerful RGB-band feature representations of EfficientSAM, and allow the original backbone branch to be fine-tuned for the current task. To enhance the adaptability of our PlaneSAM to the RGB-D domain, we pretrain our dual-complexity backbone using the segment anything task on large-scale RGB-D data through a self-supervised pretraining strategy based on imperfect pseudo-labels. To support the segmentation of large planes, we optimize the loss function combination ratio of EfficientSAM. In addition, Faster R-CNN is used as a plane detector, and its predicted bounding boxes are fed into our dual-complexity network as prompts, thereby enabling fully automatic plane instance segmentation. Experimental results show that the proposed PlaneSAM sets a new SOTA performance on the ScanNet dataset, and outperforms previous SOTA approaches in zero-shot transfer on the 2D-3D-S, Matterport3D, and ICL-NUIM RGB-D datasets, while only incurring a 10% increase in computational overhead compared to EfficientSAM., Comment: submitted to Information Fusion

Live comments, also known as Danmaku, are user-generated messages that are synchronized with video content. These comments overlay directly onto streaming videos, capturing viewer emotions and reactions in real-time. While prior work has leveraged live comments in affective analysis, its use has been limited due to the relative rarity of live comments across different video platforms. To address this, we first construct the Live Comment for Affective Analysis (LCAffect) dataset which contains live comments for English and Chinese videos spanning diverse genres that elicit a wide spectrum of emotions. Then, using this dataset, we use contrastive learning to train a video encoder to produce synthetic live comment features for enhanced multimodal affective content analysis. Through comprehensive experimentation on a wide range of affective analysis tasks (sentiment, emotion recognition, and sarcasm detection) in both English and Chinese, we demonstrate that these synthetic live comment features significantly improve performance over state-of-the-art methods.

Spherical proportional counters (SPCs) are gaseous particle detectors sensitive to single ionization electrons in their target media, with large detector volumes and low background rates. The $\mbox{NEWS-G}$ collaboration employs this technology to search for low-mass dark matter, having previously performed searches with detectors at the Laboratoire Souterrain de Modane (LSM), including a recent campaign with a 135 cm diameter SPC filled with methane. While in situ calibrations of the detector response were carried out at the LSM, measurements of the mean ionization yield and fluctuations of methane gas in SPCs were performed using a 30 cm diameter detector. The results of multiple measurements taken at different operating voltages are presented. A UV laser system was used to measure the mean gas gain of the SPC, along with $\mathrm{^{37}Ar}$ and aluminum-fluorescence calibration sources. These measurements will inform the energy response model of future operating detectors., Comment: 11 pages, 8 figures

StarCraft Multi-Agent Challenge (SMAC) is one of the most commonly used experimental environments in multi-agent reinforcement learning (MARL), where the specific task is to control a set number of allied units to defeat enemy forces. Traditional MARL algorithms often require interacting with the environment for up to 1 million steps to train a model, and the resulting policies are typically non-interpretable with weak transferability. In this paper, we propose a novel approach to solving SMAC tasks called LLM-SMAC. In our framework, agents leverage large language models (LLMs) to generate decision tree code by providing task descriptions. The model is further self-reflection using feedback from the rewards provided by the environment. We conduct experiments in the SMAC and demonstrate that our method can produce high-quality, interpretable decision trees with minimal environmental exploration. Moreover, these models exhibit strong transferability, successfully applying to similar SMAC environments without modification. We believe this approach offers a new direction for solving decision-making tasks in the future.

In the pursuit of quantum computing, solid-state quantum systems, particularly superconducting ones, have made remarkable advancements over the past two decades. However, achieving fault-tolerant quantum computing for next-generation applications necessitates the integration of several million qubits, which presents significant challenges in terms of interconnection complexity and latency that are currently unsolvable with state-of-the-art room-temperature control and readout electronics. Recently, cryogenic integrated circuits (ICs), including CMOS radio-frequency ICs and rapid-single-flux-quantum-logic ICs, have emerged as potential alternatives to room-temperature electronics. Unlike their room-temperature counterparts, these ICs are deployed within cryostats to enhance scalability by reducing the number and length of transmission lines. Additionally, operating at cryogenic temperatures can suppress electronic noise and improve qubit control fidelity. However, for CMOS ICs specifically, circuit design uncertainties arise due to a lack of reliable models for cryogenic field effect transistors as well as issues related to severe fickle noises and power dissipation at cryogenic temperatures. This paper provides a comprehensive review of recent research on both types of cryogenic control and readout ICs but primarily focuses on the more mature CMOS technology. The discussion encompasses principles underlying control and readout techniques employed in cryogenic CMOS ICs along with their architectural designs; characterization and modeling approaches for field effect transistors under cryogenic conditions; as well as fundamental concepts pertaining to rapid single flux quantum circuits.

Unmanned Aerial Vehicle (UAV) swarms are increasingly deployed in dynamic, data-rich environments for applications such as environmental monitoring and surveillance. These scenarios demand efficient data processing while maintaining privacy and security, making Federated Learning (FL) a promising solution. FL allows UAVs to collaboratively train global models without sharing raw data, but challenges arise due to the non-Independent and Identically Distributed (non-IID) nature of the data collected by UAVs. In this study, we show an integration of the state-of-the-art FL methods to UAV Swarm application and invetigate the performance of multiple aggregation methods (namely FedAvg, FedProx, FedOpt, and MOON) with a particular focus on tackling non-IID on a variety of datasets, specifically MNIST for baseline performance, CIFAR10 for natural object classification, EuroSAT for environment monitoring, and CelebA for surveillance. These algorithms were selected to cover improved techniques on both client-side updates and global aggregation. Results show that while all algorithms perform comparably on IID data, their performance deteriorates significantly under non-IID conditions. FedProx demonstrated the most stable overall performance, emphasising the importance of regularising local updates in non-IID environments to mitigate drastic deviations in local models.

Reliable object grasping is one of the fundamental tasks in robotics. However, determining grasping pose based on single-image input has long been a challenge due to limited visual information and the complexity of real-world objects. In this paper, we propose Triplane Grasping, a fast grasping decision-making method that relies solely on a single RGB-only image as input. Triplane Grasping creates a hybrid Triplane-Gaussian 3D representation through a point decoder and a triplane decoder, which produce an efficient and high-quality reconstruction of the object to be grasped to meet real-time grasping requirements. We propose to use an end-to-end network to generate 6-DoF parallel-jaw grasp distributions directly from 3D points in the point cloud as potential grasp contacts and anchor the grasp pose in the observed data. Experiments demonstrate that our method achieves rapid modeling and grasping pose decision-making for daily objects, and exhibits a high grasping success rate in zero-shot scenarios.

Vision-Language Models (VLMs) play a crucial role in robotic manipulation by enabling robots to understand and interpret the visual properties of objects and their surroundings, allowing them to perform manipulation based on this multimodal understanding. However, understanding object attributes and spatial relationships is a non-trivial task but is critical in robotic manipulation tasks. In this work, we present a new dataset focused on spatial relationships and attribute assignment and a novel method to utilize VLMs to perform object manipulation with task-oriented, high-level input. In this dataset, the spatial relationships between objects are manually described as captions. Additionally, each object is labeled with multiple attributes, such as fragility, mass, material, and transparency, derived from a fine-tuned vision language model. The embedded object information from captions are automatically extracted and transformed into a data structure (in this case, tree, for demonstration purposes) that captures the spatial relationships among the objects within each image. The tree structures, along with the object attributes, are then fed into a language model to transform into a new tree structure that determines how these objects should be organized in order to accomplish a specific (high-level) task. We demonstrate that our method not only improves the comprehension of spatial relationships among objects in the visual environment but also enables robots to interact with these objects more effectively. As a result, this approach significantly enhances spatial reasoning in robotic manipulation tasks. To our knowledge, this is the first method of its kind in the literature, offering a novel solution that allows robots to more efficiently organize and utilize objects in their surroundings.

Reconstructing 3D clothed humans from monocular camera data is highly challenging due to viewpoint limitations and image ambiguity. While implicit function-based approaches, combined with prior knowledge from parametric models, have made significant progress, there are still two notable problems. Firstly, the back details of human models are ambiguous due to viewpoint invisibility. The quality of the back details depends on the back normal map predicted by a convolutional neural network (CNN). However, the CNN lacks global information awareness for comprehending the back texture, resulting in excessively smooth back details. Secondly, a single image suffers from local ambiguity due to lighting conditions and body movement. However, implicit functions are highly sensitive to pixel variations in ambiguous regions. To address these ambiguities, we propose the Spatial-Temporal Transformer (STT) network for 3D clothed human reconstruction. A spatial transformer is employed to extract global information for normal map prediction. The establishment of global correlations facilitates the network in comprehending the holistic texture and shape of the human body. Simultaneously, to compensate for local ambiguity in images, a temporal transformer is utilized to extract temporal features from adjacent frames. The incorporation of temporal features can enhance the accuracy of input features in implicit networks. Furthermore, to obtain more accurate temporal features, joint tokens are employed to establish local correspondences between frames. Experimental results on the Adobe and MonoPerfCap datasets have shown that our method outperforms state-of-the-art methods and maintains robust generalization even under low-light outdoor conditions.

Large Language Models (LLMs) are thought to struggle with arithmetic learning due to the inherent differences between language modeling and numerical computation, but concrete evidence has been lacking. This work responds to this claim through a two-side experiment. We first investigate whether LLMs leverage partial products during arithmetic learning. We find that although LLMs can identify some partial products after learning, they fail to leverage them for arithmetic tasks, conversely. We then explore how LLMs approach arithmetic symbolically by breaking tasks into subgroups, hypothesizing that difficulties arise from subgroup complexity and selection. Our results show that when subgroup complexity is fixed, LLMs treat a collection of different arithmetic operations similarly. By analyzing position-level accuracy across different training sizes, we further observe that it follows a U-shaped pattern: LLMs quickly learn the easiest patterns at the first and last positions, while progressively learning the more difficult patterns in the middle positions. This suggests that LLMs select subgroup following an easy-to-hard paradigm during learning. Our work confirms that LLMs are pure symbolic learners in arithmetic tasks and underscores the importance of understanding them deeply through subgroup-level quantification.

The accurate segmentation of retinal vessels in fundus images is a great challenge in medical image segmentation tasks due to their highly complex structure from other organs.Currently, deep-learning based methods for retinal cessel segmentation achieved suboptimal outcoms,since vessels with indistinct features are prone to being overlooked in deeper layers of the network. Additionally, the abundance of redundant information in the background poses significant interference to feature extraction, thus increasing the segmentation difficulty. To address this issue, this paper proposes a feature-enhanced interaction network based on DPCN, named MDFI-Net.Specifically, we design a feature enhancement structure, the Deformable-convolutional Pulse Coupling Network (DPCN), to provide an enhanced feature iteration sequence to the segmentation network in a simple and efficient manner. Subsequently, these features will interact within the segmentation network.Extensive experiments were conducted on publicly available retinal vessel segmentation datasets to validate the effectiveness of our network structure. Experimental results of our algorithm show that the detection accuracy of the retinal blood vessel achieves 97.91%, 97.97% and 98.16% across all datasets. Finally, plentiful experimental results also prove that the proposed MDFI-Net achieves segmentation performance superior to state-of-the-art methods on public datasets.

We propose Counterfactual Analysis Quadratic Unconstrained Binary Optimization (CAQUBO) to solve QUBO problems for feature selection in recommender systems. CAQUBO leverages counterfactual analysis to measure the impact of individual features and feature combinations on model performance and employs the measurements to construct the coefficient matrix for a quantum annealer to select the optimal feature combinations for recommender systems, thereby improving their final recommendation performance. By establishing explicit connections between features and the recommendation performance, the proposed approach demonstrates superior performance compared to the state-of-the-art quantum annealing methods. Extensive experiments indicate that integrating quantum computing with counterfactual analysis holds great promise for addressing these challenges.

Mesh smoothing methods can enhance mesh quality by eliminating distorted elements, leading to improved convergence in simulations. To balance the efficiency and robustness of traditional mesh smoothing process, previous approaches have employed supervised learning and reinforcement learning to train intelligent smoothing models. However, these methods heavily rely on labeled dataset or prior knowledge to guide the models' learning. Furthermore, their limited capacity to enhance mesh connectivity often restricts the effectiveness of smoothing. In this paper, we first systematically analyze the learning mechanisms of recent intelligent smoothing methods and propose a prior-free reinforcement learning model for intelligent mesh smoothing. Our proposed model integrates graph neural networks with reinforcement learning to implement an intelligent node smoothing agent and introduces, for the first time, a mesh connectivity improvement agent. We formalize mesh optimization as a Markov Decision Process and successfully train both agents using Twin Delayed Deep Deterministic Policy Gradient and Double Dueling Deep Q-Network in the absence of any prior data or knowledge. We verified the proposed model on both 2D and 3D meshes. Experimental results demonstrate that our model achieves feature-preserving smoothing on complex 3D surface meshes. It also achieves state-of-the-art results among intelligent smoothing methods on 2D meshes and is 7.16 times faster than traditional optimization-based smoothing methods. Moreover, the connectivity improvement agent can effectively enhance the quality distribution of the mesh.

In this paper, we study learning-augmented algorithms for the Bahncard problem. The Bahncard problem is a generalization of the ski-rental problem, where a traveler needs to irrevocably and repeatedly decide between a cheap short-term solution and an expensive long-term one with an unknown future. Even though the problem is canonical, only a primal-dual-based learning-augmented algorithm was explicitly designed for it. We develop a new learning-augmented algorithm, named PFSUM, that incorporates both history and short-term future to improve online decision making. We derive the competitive ratio of PFSUM as a function of the prediction error and conduct extensive experiments to show that PFSUM outperforms the primal-dual-based algorithm., Comment: This paper has been accepted by the 38th Conference on Neural Information Processing Systems (NeurIPS 2024)

For superconducting quantum processors, stable high-fidelity two-qubit operations depend on precise flux control of the tunable coupler. However, the pulse distortion poses a significant challenge to the control precision. Current calibration methods, which often rely on microwave crosstalk or additional readout resonators for coupler excitation and readout, tend to be cumbersome and inefficient, especially when couplers only have flux control. Here, we introduce and experimentally validate a novel pulse calibration scheme that exploits the strong coupling between qubits and couplers, eliminating the need for extra coupler readout and excitation. Our method directly measures the short-time and long-time step responses of the coupler flux pulse transient, enabling us to apply predistortion to subsequent signals using fast Fourier transformation and deconvolution. This approach not only simplifies the calibration process but also significantly improves the precision and stability of the flux control. We demonstrate the efficacy of our method through the implementation of diabatic CZ and iSWAP gates with fidelities of $99.61\pm0.04\%$ and $99.82\pm0.02\%$, respectively, as well as a series of diabatic CPhase gates with high fidelities characterized by cross-entropy benchmarking. The consistency and robustness of our technique are further validated by the reduction in pulse distortion and phase error observed across multilayer CZ gates. These results underscore the potential of our calibration and predistortion method to enhance the performance of two-qubit gates in superconducting quantum processors., Comment: 16 pages, 11 figures, 7 tables

Positional bias in large language models (LLMs) hinders their ability to effectively process long inputs. A prominent example is the "lost in the middle" phenomenon, where LLMs struggle to utilize relevant information situated in the middle of the input. While prior research primarily focuses on single pieces of relevant information, real-world applications often involve multiple relevant information pieces. To bridge this gap, we present LongPiBench, a benchmark designed to assess positional bias involving multiple pieces of relevant information. Thorough experiments are conducted with five commercial and six open-source models. These experiments reveal that while most current models are robust against the "lost in the middle" issue, there exist significant biases related to the spacing of relevant information pieces. These findings highlight the importance of evaluating and reducing positional biases to advance LLM's capabilities., Comment: work in progress

Multi-modal entity alignment (MMEA) is essential for enhancing knowledge graphs and improving information retrieval and question-answering systems. Existing methods often focus on integrating modalities through their complementarity but overlook the specificity of each modality, which can obscure crucial features and reduce alignment accuracy. To solve this, we propose the Multi-modal Consistency and Specificity Fusion Framework (MCSFF), which innovatively integrates both complementary and specific aspects of modalities. We utilize Scale Computing's hyper-converged infrastructure to optimize IT management and resource allocation in large-scale data processing. Our framework first computes similarity matrices for each modality using modality embeddings to preserve their unique characteristics. Then, an iterative update method denoises and enhances modality features to fully express critical information. Finally, we integrate the updated information from all modalities to create enriched and precise entity representations. Experiments show our method outperforms current state-of-the-art MMEA baselines on the MMKG dataset, demonstrating its effectiveness and practical potential., Comment: 6 pages, 1 figures

Efficient and systematic numerical methods for robust control design are highly desired in the study of quantum systems, since system imperfections such as uncertainties or disturbances inevitably exist. By modeling uncertainties as combinations of random variables following prescribed distributions, the expectation of infidelity can be used to quantify the level of robustness as a performance index in various control problems. Inspired by sample-based algorithms, we consider the level of robustness as a weighted tensor product quadrature, and take advantage of Smolyak's sparse grid algorithm to develop a parametric robust quantum control scheme, with the purpose of reducing computation cost and improving accuracy. Furthermore, the features and strengths of the scheme proposed in this paper have been illustrated in the context of robust control problems, including state transfer and realization of quantum gates, where ultrahigh fidelity can be achieved with strong robustness using Smolyak algorithm-assisted gradient-based methods. Therefore, our results can contribute to reliable and secure quantum computing as well as communication.

We study the spatially incoherent light generated by a multimode fiber(MMF) in the application of image projection designed for the ultracold-atom experiments. Inspired by previous half-analytic methods concerning the incoherent light, here a full-numerical model is established to provide more quantitative descriptions, and part of results is compared with experiments. Particularly, our model about the MMF is also compatible with light propagation in free space. Based on this, we study both the intrinsic speckle and the perturbation robustness of a MMF light field, under the influence of light propagation and fiber parameters. We point out several guidelines about choosing the suitable MMF in creating a spatially incoherent light source, which is useful in the context of the ultracold-atom experiments associating with the optical potential projection.

The recent advancements in large language models (LLMs) and pre-trained vision models have accelerated the development of vision-language large models (VLLMs), enhancing the interaction between visual and linguistic modalities. Despite their notable success across various domains, VLLMs face challenges in modality alignment, which can lead to issues like hallucinations and unsafe content generation. Current alignment techniques often rely on coarse feedback and external datasets, limiting scalability and performance. In this paper, we propose FiSAO (Fine-Grained Self-Alignment Optimization), a novel self-alignment method that utilizes the model's own visual encoder as a fine-grained verifier to improve vision-language alignment without the need for additional data. By leveraging token-level feedback from the vision encoder, FiSAO significantly improves vision-language alignment, even surpassing traditional preference tuning methods that require additional data. Through both theoretical analysis and experimental validation, we demonstrate that FiSAO effectively addresses the misalignment problem in VLLMs, marking the first instance of token-level rewards being applied to such models., Comment: 23 pages

As the capabilities of Multimodal Large Language Models (MLLMs) continue to improve, the need for higher-order capability evaluation of MLLMs is increasing. However, there is a lack of work evaluating MLLM for higher-order perception and understanding of Chinese visual content. To fill the gap, we introduce the **C**hinese **I**mage **I**mplication understanding **Bench**mark, **CII-Bench**, which aims to assess the higher-order perception and understanding capabilities of MLLMs for Chinese images. CII-Bench stands out in several ways compared to existing benchmarks. Firstly, to ensure the authenticity of the Chinese context, images in CII-Bench are sourced from the Chinese Internet and manually reviewed, with corresponding answers also manually crafted. Additionally, CII-Bench incorporates images that represent Chinese traditional culture, such as famous Chinese traditional paintings, which can deeply reflect the model's understanding of Chinese traditional culture. Through extensive experiments on CII-Bench across multiple MLLMs, we have made significant findings. Initially, a substantial gap is observed between the performance of MLLMs and humans on CII-Bench. The highest accuracy of MLLMs attains 64.4%, where as human accuracy averages 78.2%, peaking at an impressive 81.0%. Subsequently, MLLMs perform worse on Chinese traditional culture images, suggesting limitations in their ability to understand high-level semantics and lack a deep knowledge base of Chinese traditional culture. Finally, it is observed that most models exhibit enhanced accuracy when image emotion hints are incorporated into the prompts. We believe that CII-Bench will enable MLLMs to gain a better understanding of Chinese semantics and Chinese-specific images, advancing the journey towards expert artificial general intelligence (AGI). Our project is publicly available at https://cii-bench.github.io/., Comment: 32 pages,18 figures. Project Page: https://cii-bench.github.io/ Code: https://github.com/MING_X/CII-Bench Dataset: https://huggingface.co/datasets/m-a-p/CII-Bench

Lepton flavour universality in rare $b\rightarrow s$ transitions is tested for the first time using $B_s^0$ meson decays. The measurements are performed using $pp$ collision data collected by the LHCb experiment between 2011 and 2018, corresponding to a total integrated luminosity of 9$\,{\rm fb}^{-1}$. Branching fraction ratios between the $B_s^0 \rightarrow \phi e^+e^-$ and $B_s^0 \rightarrow \phi \mu^+\mu^-$ decays are measured in three regions of dilepton mass squared, $q^2$, with $0.1 < q^2 < 1.1$, $1.1 < q^2 < 6.0$, and $15 < q^2 < 19\,{\rm GeV}^2/c^4$. The results agree with the Standard Model expectation of lepton flavour universality., Comment: All figures and tables, along with machine-readable versions and any supplementary material and additional information, are available at https://lbfence.cern.ch/alcm/public/analysis/full-details/3513/ (LHCb public pages)

Dynamic scenes rendering is an intriguing yet challenging problem. Although current methods based on NeRF have achieved satisfactory performance, they still can not reach real-time levels. Recently, 3D Gaussian Splatting (3DGS) has garnered researchers attention due to their outstanding rendering quality and real-time speed. Therefore, a new paradigm has been proposed: defining a canonical 3D gaussians and deforming it to individual frames in deformable fields. However, since the coordinates of canonical 3D gaussians are filled with noise, which can transfer noise into the deformable fields, and there is currently no method that adequately considers the aggregation of 4D information. Therefore, we propose Denoised Deformable Network with Temporal-Spatial Aggregation for Dynamic Scene Rendering (DN-4DGS). Specifically, a Noise Suppression Strategy is introduced to change the distribution of the coordinates of the canonical 3D gaussians and suppress noise. Additionally, a Decoupled Temporal-Spatial Aggregation Module is designed to aggregate information from adjacent points and frames. Extensive experiments on various real-world datasets demonstrate that our method achieves state-of-the-art rendering quality under a real-time level., Comment: Accepted by NeurIPS 2024

Semantic segmentation of remote sensing (RS) images is a challenging yet crucial task. While deep learning, particularly supervised learning with large-scale labeled datasets, has significantly advanced this field, acquiring high-quality labeled data is expensive and time-consuming. Additionally, variations in ground sampling distance, imaging equipment, and geographic differences cause domain shifts between datasets, which limit model performance across domains. Unsupervised domain adaptation (UDA) offers a solution by enabling models to learn from unlabeled target domain data while training on labeled source domain data. Recent self-supervised learning approaches using pseudo-label generation have shown promise in addressing domain discrepancies. Combining source and target images with their true and pseudo labels has proven effective in reducing domain bias. However, the use of pseudo-labeling for RS image segmentation is underexplored. Existing methods often rely on high-confidence pixel points as pseudo-labels, reducing supervision in low-confidence areas. Noise in pseudo-labels further weakens the model's ability to learn target domain semantics. While some methods assign confidence weights, noisy pseudo-labels remain an issue. To address these limitations, we propose integrating contrastive learning into UDA, enhancing the model's capacity to capture semantic information by maximizing the similarity between augmented views of the same image. This provides additional supervision to improve performance in the target domain. Extensive experiments on key RS datasets (Potsdam, Vaihingen, LoveDA) demonstrate that our SimSeg method outperforms existing approaches, achieving state-of-the-art results. Visualization and quantitative analyses confirm its superior ability to learn from the target domain. The code is available at \url{https://github.com/woldier/SiamSeg}.

In this paper, we present the first multi-modal FHE accelerator based on a unified architecture, which efficiently supports CKKS, TFHE, and their conversion scheme within a single accelerator. To achieve this goal, we first analyze the theoretical foundations of the aforementioned schemes and highlight their composition from a finite number of arithmetic kernels. Then, we investigate the challenges for efficiently supporting these kernels within a unified architecture, which include 1) concurrent support for NTT and FFT, 2) maintaining high hardware utilization across various polynomial lengths, and 3) ensuring consistent performance across diverse arithmetic kernels. To tackle these challenges, we propose a novel FHE accelerator named Trinity, which incorporates algorithm optimizations, hardware component reuse, and dynamic workload scheduling to enhance the acceleration of CKKS, TFHE, and their conversion scheme. By adaptive select the proper allocation of components for NTT and MAC, Trinity maintains high utilization across NTTs with various polynomial lengths and imbalanced arithmetic workloads. The experiment results show that, for the pure CKKS and TFHE workloads, the performance of our Trinity outperforms the state-of-the-art accelerator for CKKS (SHARP) and TFHE (Morphling) by 1.49x and 4.23x, respectively. Moreover, Trinity achieves 919.3x performance improvement for the FHE-conversion scheme over the CPU-based implementation. Notably, despite the performance improvement, the hardware overhead of Trinity is only 85% of the summed circuit areas of SHARP and Morphling., Comment: To be appeared in MICRO 2024. The first ASIC-based FHE accelerator which supports both CKKS, TFHE and their conversions. Provide new SOTA performance record for CKKS, TFHE and conversion

The cloud cover and meteorological parameters serve as fundamental criteria for the qualification of an astronomical observatory working in optical and infrared wavelengths. In this paper, we present a systematic assessment of key meteorological parameters at the Lenghu site. The datasets adopted in this study includes the meteorological parameters collected at the local weather stations at the site and in the Lenghu Town, the sky brightness at the local zenith acquired by the Sky Quality Meters and night sky all-sky images from a digital camera, the ERA5 reanalysis database and global climate monitoring data. From 2019 to 2023, the fractional observable time of photometric condition is 69.70%, 74.97%, 70.26%, 74.27% and 65.12%, respectively. The fractional observing time is inversely correlated with surface air temperature, relative humidity, precipitable water vapor, and dew temperature, demonstrating that the observing conditions are influenced by these meteorological parameters. Large-scale air-sea interactions affect the climate at Lenghu site, which in fact delivers a clue to understand the irregularity of 2023. Specifically, precipitable water vapor at Lenghu site is correlated to both the westerly wind index and the summer North Atlantic Oscillation index, the yearly average temperature of Lenghu site is observed to increase significantly during the occurrence of a strong El Ni\~no event and the relative humidity anomaly at Lenghu site is correlated to the Pacific Decadal Oscillation index. The decrease of fractional observing time in 2023 was due to the ongoing strong El Ni\~no event and relevant global climate change. We underscore the substantial role of global climate change in regulating astronomical observing conditions and the necessity for long-term continuous monitoring of the astronomical meteorological parameters at Lenghu site., Comment: accepted for publication in MNRAS

Edge Intelligence (EI) has been instrumental in delivering real-time, localized services by leveraging the computational capabilities of edge networks. The integration of Large Language Models (LLMs) empowers EI to evolve into the next stage: Edge General Intelligence (EGI), enabling more adaptive and versatile applications that require advanced understanding and reasoning capabilities. However, systematic exploration in this area remains insufficient. This survey delineates the distinctions between EGI and traditional EI, categorizing LLM-empowered EGI into three conceptual systems: centralized, hybrid, and decentralized. For each system, we detail the framework designs and review existing implementations. Furthermore, we evaluate the performance and throughput of various Small Language Models (SLMs) that are more suitable for development on edge devices. This survey provides researchers with a comprehensive vision of EGI, offering insights into its vast potential and establishing a foundation for future advancements in this rapidly evolving field.

KV cache has become a de facto technique for the inference of large language models (LLMs), where tensors of shape (layer number, head number, sequence length, feature dimension) are introduced to cache historical information for self-attention. As the size of the model and data grows, the KV cache can quickly become a bottleneck within the system in both storage and memory transfer. To address this, prior studies usually focus on the first three axes of the cache tensors for compression. This paper supplements them, focusing on the feature dimension axis, by utilizing low-rank projection matrices to transform the cache features into spaces with reduced dimensions. We begin by investigating the canonical orthogonal projection method for data compression through principal component analysis (PCA). We observe the issue with PCA projection where significant performance degradation is observed at low compression rates. To bridge the gap, we propose to directly tune the orthogonal projection matrices with a distillation objective using an elaborate Matryoshka training strategy. After training, we adaptively search for the optimal compression rates for various layers and heads given varying compression budgets. Compared to previous works, our method can easily embrace pre-trained LLMs and hold a smooth tradeoff between performance and compression rate. We empirically witness the high data efficiency of our training procedure and find that our method can sustain over 90% performance with an average KV cache compression rate of 60% (and up to 75% in certain extreme scenarios) for popular LLMs like LLaMA2-7B-base and Mistral-7B-v0.3-base.

Functional safety is a critical aspect of automotive engineering, encompassing all phases of a vehicle's lifecycle, including design, development, production, operation, and decommissioning. This domain involves highly knowledge-intensive tasks. This paper introduces Aegis: An Advanced LLM-Based Multi-Agent for Intelligent Functional Safety Engineering. Aegis is specifically designed to support complex functional safety tasks within the automotive sector. It is tailored to perform Hazard Analysis and Risk Assessment(HARA), document Functional Safety Requirements(FSR), and plan test cases for Automatic Emergency Braking(AEB) systems. The most advanced version, Aegis-Max, leverages Retrieval-Augmented Generation(RAG) and reflective mechanisms to enhance its capability in managing complex, knowledge-intensive tasks. Additionally, targeted prompt refinement by professional functional safety practitioners can significantly optimize Aegis's performance in the functional safety domain. This paper demonstrates the potential of Aegis to improve the efficiency and effectiveness of functional safety processes in automotive engineering.

Recommender systems are extensively utilised across various areas to predict user preferences for personalised experiences and enhanced user engagement and satisfaction. Traditional recommender systems, however, are complicated by confounding bias, particularly in the presence of latent confounders that affect both item exposure and user feedback. Existing debiasing methods often fail to capture the complex interactions caused by latent confounders in interaction data, especially when dual latent confounders affect both the user and item sides. To address this, we propose a novel debiasing method that jointly integrates the Instrumental Variables (IV) approach and identifiable Variational Auto-Encoder (iVAE) for Debiased representation learning in Recommendation systems, referred to as IViDR. Specifically, IViDR leverages the embeddings of user features as IVs to address confounding bias caused by latent confounders between items and user feedback, and reconstructs the embedding of items to obtain debiased interaction data. Moreover, IViDR employs an Identifiable Variational Auto-Encoder (iVAE) to infer identifiable representations of latent confounders between item exposure and user feedback from both the original and debiased interaction data. Additionally, we provide theoretical analyses of the soundness of using IV and the identifiability of the latent representations. Extensive experiments on both synthetic and real-world datasets demonstrate that IViDR outperforms state-of-the-art models in reducing bias and providing reliable recommendations.

Large language models (LLMs) have become increasingly proficient at simulating various personality traits, an important capability for supporting related applications (e.g., role-playing). To further improve this capacity, in this paper, we present a neuron-based approach for personality trait induction in LLMs, with three major technical contributions. First, we construct PersonalityBench, a large-scale dataset for identifying and evaluating personality traits in LLMs. This dataset is grounded in the Big Five personality traits from psychology and is designed to assess the generative capabilities of LLMs towards specific personality traits. Second, by leveraging PersonalityBench, we propose an efficient method for identifying personality-related neurons within LLMs by examining the opposite aspects of a given trait. Third, we develop a simple yet effective induction method that manipulates the values of these identified personality-related neurons. This method enables fine-grained control over the traits exhibited by LLMs without training and modifying model parameters. Extensive experiments validate the efficacy of our neuron identification and trait induction methods. Notably, our approach achieves comparable performance as fine-tuned models, offering a more efficient and flexible solution for personality trait induction in LLMs. We provide access to all the mentioned resources at https://github.com/RUCAIBox/NPTI.

Weyl semimetal (WSM) states can be achieved by breaking spatial-inversion symmetry or time reversal symmetry. However, the anisotropy of the energy reduction contributing to the emergence of WSM states has seldom been investigated by experiments. A van der Waals metal MoTe2 exhibits a type-II WSM phase below the monoclinic-to-orthorhombic-phase-transition temperature Tc ~ 250 K. Here, we report a combined linearly-polarized optical-spectroscopy and electrical-transport study of MoTe2 at different temperatures. The Drude components in the a-axis, b-axis and c-axis optical conductivity spectra, together with the metallic out-of-plane and in-plane electrical resistivities, indicate the coherent inter-layer and in-plane charge transports. Moreover, the Drude weight in {\sigma}1a({\omega}), rather than the Drude weights in {\sigma}1b({\omega}) and {\sigma}1c({\omega}), decreases dramatically below Tc, which exhibits a highly anisotropic decrease in its Drude weight and thus suggests a strongly anisotropic reduction of the electronic kinetic energy in the WSM phase. Furthermore, below Tc, due to the in-plane anisotropic spectral-weight transfer from Drude component to high-energy region, the in-plane inter-band-absorption anisotropy increases remarkably around 770 meV, and has the largest value (~ 0.68) of normalized linear dichroism among the reported type-II WSMs. Our work sheds light on seeking new WSMs and developing novel photonic devices based on WSMs., Comment: Accepted by Laser & Photonics Reviews

Interactive segmentation aims to accurately segment target objects with minimal user interactions. However, current methods often fail to accurately separate target objects from the background, due to a limited understanding of order, the relative depth between objects in a scene. To address this issue, we propose OIS: order-aware interactive segmentation, where we explicitly encode the relative depth between objects into order maps. We introduce a novel order-aware attention, where the order maps seamlessly guide the user interactions (in the form of clicks) to attend to the image features. We further present an object-aware attention module to incorporate a strong object-level understanding to better differentiate objects with similar order. Our approach allows both dense and sparse integration of user clicks, enhancing both accuracy and efficiency as compared to prior works. Experimental results demonstrate that OIS achieves state-of-the-art performance, improving mIoU after one click by 7.61 on the HQSeg44K dataset and 1.32 on the DAVIS dataset as compared to the previous state-of-the-art SegNext, while also doubling inference speed compared to current leading methods. The project page is https://ukaukaaaa.github.io/projects/OIS/index.html, Comment: Interactive demo can be found in project page: https://ukaukaaaa.github.io/projects/OIS/index.html

With the rapid progress of diffusion-based content generation, significant efforts are being made to unlearn harmful or copyrighted concepts from pretrained diffusion models (DMs) to prevent potential model misuse. However, it is observed that even when DMs are properly unlearned before release, malicious finetuning can compromise this process, causing DMs to relearn the unlearned concepts. This occurs partly because certain benign concepts (e.g., "skin") retained in DMs are related to the unlearned ones (e.g., "nudity"), facilitating their relearning via finetuning. To address this, we propose meta-unlearning on DMs. Intuitively, a meta-unlearned DM should behave like an unlearned DM when used as is; moreover, if the meta-unlearned DM undergoes malicious finetuning on unlearned concepts, the related benign concepts retained within it will be triggered to self-destruct, hindering the relearning of unlearned concepts. Our meta-unlearning framework is compatible with most existing unlearning methods, requiring only the addition of an easy-to-implement meta objective. We validate our approach through empirical experiments on meta-unlearning concepts from Stable Diffusion models (SD-v1-4 and SDXL), supported by extensive ablation studies. Our code is available at https://github.com/sail-sg/Meta-Unlearning.

Nonreciprocal electrical transport, characterized by an asymmetric relationship between current and voltage, plays a crucial role in modern electronic industries. Recent studies have extended this phenomenon to superconductors, introducing the concept of the superconducting diode effect (SDE). The SDE is characterized by unequal critical supercurrents along opposite directions. Due to the requirement on broken inversion symmetry, the SDE is commonly accompanied by electrical magnetochiral anisotropy (eMCA) in the resistive state. Achieving a magnetic field-free SDE with field tunability is pivotal for advancements in superconductor devices. Conventionally, the field-free SDE has been achieved in Josephson junctions by intentionally intercalating an asymmetric barrier layer. Alternatively, internal magnetism was employed. Both approaches pose challenges in the selection of superconductors and fabrication processes, thereby impeding the development of SDE. Here, we present a field-free SDE in FeTe0.7Se0.3 (FTS) junction with eMCA, a phenomenon absent in FTS single nanosheets. The field-free property is associated with the presence of a gradient oxide layer on the upper surface of each FTS nanosheet, while the eMCA is linked to spin-splitting arising from the absence of inversion symmetry. Both the SDE and eMCA respond to magnetic fields with distinct temperature dependencies. This work presents a versatile and straightforward strategy for advancing superconducting electronics.

Model merging has become one of the key technologies for enhancing the capabilities and efficiency of Large Language Models (LLMs). However, our understanding of the expected performance gains and principles when merging any two models remains limited. In this work, we introduce model kinship, the degree of similarity or relatedness between LLMs, analogous to biological evolution. With comprehensive empirical analysis, we find that there is a certain relationship between model kinship and the performance gains after model merging, which can help guide our selection of candidate models. Inspired by this, we propose a new model merging strategy: Top-k Greedy Merging with Model Kinship, which can yield better performance on benchmark datasets. Specifically, we discover that using model kinship as a criterion can assist us in continuously performing model merging, alleviating the degradation (local optima) in model evolution, whereas model kinship can serve as a guide to escape these traps. Code is available at https://github.com/zjunlp/ModelKinship., Comment: Ongoing work

Subwavelength grating micro-polarizer arrays, as a type of focal plane division simultaneous detection method, are significantly advancing the development and practical application of polarization imaging technology. Based on the cross-scale, dual-period characteristics of the grating array, this paper proposes a fabrication method that combines laser interference lithography with contact lithography. This method constructs a complete single-layer micro-polarizer array photoresist pattern through a four-step lithography process. Compared to traditional point-by-point fabrication methods like EBL and FIB, the patterning time is reduced by 3 to 4 orders of magnitude. Additionally, by introducing a refractive index matching liquid and an alignment method based on substrate contours, the effects of gaps and splicing errors are minimized, resulting in high-quality photoresist patterns with splicing errors less than 1{\mu}m. Finally, a double-layer metal grating structure was obtained through pattern transfer. Performance tests show that the micro-polarizer array achieves a maximum transmittance of over 50% and an extinction ratio exceeding 15dB in the 3-15{\mu}m wavelength range. This exploration offers a low-cost, high-efficiency path for fabricating micro-polarizer arrays., Comment: 23 pages, 13figures

Spatial Database Management Systems (SDBMSs) aim to store, manipulate, and retrieve spatial data. SDBMSs are employed in various modern applications, such as geographic information systems, computer-aided design tools, and location-based services. However, the presence of logic bugs in SDBMSs can lead to incorrect results, substantially undermining the reliability of these applications. Detecting logic bugs in SDBMSs is challenging due to the lack of ground truth for identifying incorrect results. In this paper, we propose an automated geometry-aware generator to generate high-quality SQL statements for SDBMSs and a novel concept named Affine Equivalent Inputs (AEI) to validate the results of SDBMSs. We implemented them as a tool named Spatter (Spatial DBMSs Tester) for finding logic bugs in four popular SDBMSs: PostGIS, DuckDB Spatial, MySQL, and SQL Server. Our testing campaign detected 34 previously unknown and unique bugs in these SDBMS, of which 30 have been confirmed, and 18 have been already fixed. Our testing efforts have been well appreciated by the developers. Experimental results demonstrate that the geometry-aware generator significantly outperforms a naive random-shape generator in detecting unique bugs, and AEI can identify 14 logic bugs in SDBMSs that were overlooked by previous methodologies.

Federated Graph Neural Network (FedGNN) is a privacy-preserving machine learning technology that combines federated learning (FL) and graph neural networks (GNNs). It offers a privacy-preserving solution for training GNNs using isolated graph data. Vertical Federated Graph Neural Network (VFGNN) is an important branch of FedGNN, where data features and labels are distributed among participants, and each participant has the same sample space. Due to the difficulty of accessing and modifying distributed data and labels, the vulnerability of VFGNN to backdoor attacks remains largely unexplored. In this context, we propose BVG, the first method for backdoor attacks in VFGNN. Without accessing or modifying labels, BVG uses multi-hop triggers and requires only four target class nodes for an effective backdoor attack. Experiments show that BVG achieves high attack success rates (ASR) across three datasets and three different GNN models, with minimal impact on main task accuracy (MTA). We also evaluate several defense methods, further validating the robustness and effectiveness of BVG. This finding also highlights the need for advanced defense mechanisms to counter sophisticated backdoor attacks in practical VFGNN applications.

We perform a theoretical study on the spectrum of the low-lying hidden-bottom ($q\bar{q}b\bar{b}$ with $q=$ $u$, $d$ and $s$) and double-bottom ($q\bar{q}bb$) tetraquark states within a nonrelativistic quark model, in which the instanton-induced interaction is taken as the residual spin-dependent hyperfine interaction between quarks. All the model parameters are fixed by fitting the spectrum of the ground hadron states. The numerical results indicate that masses of several $X_{q\bar{q}}$, $Z_{b}$, and $Z_{bs}$ tetraquark states are below and near thresholds of corresponding meson-meson channels, thus these states may form components of exotic meson states with reasonable probabilities. Especially, in present model, masses of several obtained states with quantum number $I^G(J^P)=1^+(1^{+})$ are close to $Z_b^\pm(10610)$, so one may expect these states to be non-negligible components of the experimentally observed $Z_{b}$ states. Concerning to the double-bottom tetraquark states, the present results are in general consistent with other previous works. Two possible stable $T_{bb}$ states with quark content $bb\bar{n}\bar{n}$ ($n=u$ or $d$ quark) lying at energies $10558$ MeV and $10650$ MeV are found, and one possible stable $bb\bar{n}\bar{s}$ state is found, whose energy is $\sim10687$ MeV., Comment: 15 pages, 7 figures, 12 tables

In recent years, studies have demonstrated that the grain boundary (GB) migration is a three-dimensional (3D) process, characterized by a 3D mobility tensor. In this study, we develop a 3D interface random walk theory to extract the GB mobility and shear coupling tensors based on the random walk of the GB position. Using this approach, we mathematically prove the symmetry of the GB mobility tensor in the case of overdamped GB migration. The theory and its conclusions align with molecular dynamics simulation results, and the computed shear coupling aligns closely with predictions from disconnection analysis. Additionally, we refined the fast adapted random walk (FAIRWalk) method, enabling efficient extraction of the GB mobility tensor from fewer simulations while maintaining high accuracy. Building on this advancement, we conducted an extensive survey of the mobility, shear coupling, and activation energy for the migration of 388 coincidence-site lattice Ni GBs in the Olmsted database. Several intriguing phenomena were observed, including temperature-induced sudden emergence, disappearance, or inversion of shear coupling; GBs with "zero" normal mobility but high shear mobility; and a non-linear relationship between activation energy and mobility. These findings challenge the traditional understanding of GB migration and warrant further investigation.

Differential-elimination algorithms apply a finite number of differentiations and eliminations to systems of partial differential equations. For systems that are polynomially nonlinear with rational number coefficients, they guarantee the inclusion of missing integrability conditions and the statement of of existence and uniqueness theorems for local analytic solutions of such systems. Further, they are useful in obtaining systems in a form more amenable to exact and approximate solution methods. Maple's \maple{dsolve} and \maple{pdsolve} algorithms for solving PDE and ODE often automatically call such routines during applications. Indeed, even casual users of Maple's dsolve and pdsolve commands have probably unknowingly used Maple's differential-elimination algorithms. Suppose that a system of PDE has been reduced by differential-elimination to a system whose automatic existence and uniqueness algorithm has been determined to be finite-dimensional. We present an algorithm for rewriting the output as a system of parameterized ODE. Exact methods and numerical methods for solving ODE and DAE can be applied to this form.

The KV-Cache technique has become the standard for the inference of large language models (LLMs). It caches states of self-attention to avoid recomputation. Yet, it is widely criticized that KV-Cache can become a bottleneck of the LLM inference system, especially when confronted with ultra-large models and long-context queries. A natural remedy is to discard the KV-Cache for less important tokens, with StreamingLLM as an example, but the used static eviction strategies cannot flexibly adapt to varying contexts. Remedies like H2O leverage accumulative attention scores to perform dynamic eviction but suffer from the attention bias issue in capturing contextual information. This paper bridges this gap by devising a parameterized KV-Cache eviction mechanism, dubbed as Attention-Gate, which accepts the whole context as input and yields eviction flags for each token to realize in-context eviction. The subsequent self-attention module proceeds according to the flags and only the KV states for the remaining tokens need to be cached. The Attention-Gates can vary among different heads and layers and be trivially plugged into pre-trained LLMs, tuned by cost-effective continual pre-training or supervised fine-tuning objectives to acquire what to discard. The computational and memory overhead introduced by Attention-Gates is minimal. Our method is validated across multiple tasks, demonstrating both efficiency and adaptability. After a highly efficient continual pre-training, it achieves higher average accuracy and evicts more tokens compared to traditional training-free methods. In supervised fine-tuning, it not only evicts many tokens but also outperforms LoRA-finetuned LLMs on some datasets, such as RTE, where it improves accuracy by 13.9% while evicting 62.8% of tokens, showing that effective eviction of redundant tokens can even enhance performance.

Portfolio construction has been a long-standing topic of research in finance. The computational complexity and the time taken both increase rapidly with the number of investments in the portfolio. It becomes difficult, even impossible for classic computers to solve. Quantum computing is a new way of computing which takes advantage of quantum superposition and entanglement. It changes how such problems are approached and is not constrained by some of the classic computational complexity. Studies have shown that quantum computing can offer significant advantages over classical computing in many fields. The application of quantum computing has been constrained by the unavailability of actual quantum computers. In the past decade, there has been the rapid development of the large-scale quantum computer. However, software development for quantum computing is slow in many fields. In our study, we apply quantum computing to a multi-asset portfolio simulation. The simulation is based on historic data, covariance, and expected returns, all calculated using quantum computing. Although technically a solvable problem for classical computing, we believe the software development is important to the future application of quantum computing in finance. We conducted this study through simulation of a quantum computer and the use of Rensselaer Polytechnic Institute's IBM quantum computer.

Multimodal Large Language Models (MLLMs) frequently exhibit hallucination phenomena, but the underlying reasons remain poorly understood. In this paper, we present an empirical analysis and find that, although MLLMs incorrectly generate the objects in the final output, they are actually able to recognize visual objects in the preceding layers. We speculate that this may be due to the strong knowledge priors of the language model suppressing the visual information, leading to hallucinations. Motivated by this, we propose a novel dynamic correction decoding method for MLLMs (DeCo), which adaptively selects the appropriate preceding layers and proportionally integrates knowledge into the final layer to adjust the output logits. Note that DeCo is model agnostic and can be seamlessly incorporated with various classic decoding strategies and applied to different MLLMs. We evaluate DeCo on widely-used benchmarks, demonstrating that it can reduce hallucination rates by a large margin compared to baselines, highlighting its potential to mitigate hallucinations. Code is available at https://github.com/zjunlp/DeCo., Comment: Ongoing work

Real-world datasets follow an imbalanced distribution, which poses significant challenges in rare-category object detection. Recent studies tackle this problem by developing re-weighting and re-sampling methods, that utilise the class frequencies of the dataset. However, these techniques focus solely on the frequency statistics and ignore the distribution of the classes in image space, missing important information. In contrast to them, we propose FRActal CALibration (FRACAL): a novel post-calibration method for long-tailed object detection. FRACAL devises a logit adjustment method that utilises the fractal dimension to estimate how uniformly classes are distributed in image space. During inference, it uses the fractal dimension to inversely downweight the probabilities of uniformly spaced class predictions achieving balance in two axes: between frequent and rare categories, and between uniformly spaced and sparsely spaced classes. FRACAL is a post-processing method and it does not require any training, also it can be combined with many off-the-shelf models such as one-stage sigmoid detectors and two-stage instance segmentation models. FRACAL boosts the rare class performance by up to 8.6% and surpasses all previous methods on LVIS dataset, while showing good generalisation to other datasets such as COCO, V3Det and OpenImages. The code will be released.

Large Language Models (LLMs) have displayed massive improvements in reasoning and decision-making skills and can hold natural conversations with users. Recently, many tool-use benchmark datasets have been proposed. However, existing datasets have the following limitations: (1). Insufficient evaluation scenarios (e.g., only cover limited tool-use scenes). (2). Extensive evaluation costs (e.g., GPT API costs). To address these limitations, in this work, we propose a multi-granularity tool-use benchmark for large language models called MTU-Bench. For the "multi-granularity" property, our MTU-Bench covers five tool usage scenes (i.e., single-turn and single-tool, single-turn and multiple-tool, multiple-turn and single-tool, multiple-turn and multiple-tool, and out-of-distribution tasks). Besides, all evaluation metrics of our MTU-Bench are based on the prediction results and the ground truth without using any GPT or human evaluation metrics. Moreover, our MTU-Bench is collected by transforming existing high-quality datasets to simulate real-world tool usage scenarios, and we also propose an instruction dataset called MTU-Instruct data to enhance the tool-use abilities of existing LLMs. Comprehensive experimental results demonstrate the effectiveness of our MTU-Bench. Code and data will be released at https: //github.com/MTU-Bench-Team/MTU-Bench.git.

Instruction tuning helps improve pretrained large language models (LLMs) in terms of the responsiveness to human instructions, which is benefited from diversified instruction data. Federated learning extends the sources of instruction data by exploiting the diversified client-side data, making it increasingly popular for tuning LLMs. Existing approaches of federated LLM tuning typically traverse all local data during local training, bringing excessive computation overhead and posing a risk of overfitting local data. Thus, a federated data-efficient instruction tuning approach, which consumes relatively little data from the entire dataset, is needed. In response, this work introduces an approach of federated data-efficient instruction tuning for LLMs, FedHDS, which utilizes a representative subset of edge-side data, coreset, to tune the LLM. It reduces the redundancy of data samples at both intra-client and inter-client levels through a hierarchical data selection framework performed by jointly selecting a small number of representative data samples for local training without sharing the raw data. Extensive experiments conducted across six scenarios with various LLMs, datasets and data partitions demonstrate that FedHDS significantly reduces the amount of data required for fine-tuning while improving the responsiveness of the instruction-tuned LLMs to unseen tasks., Comment: 11 pages. Ongoing work

Building a foundation model for 3D vision is a complex challenge that remains unsolved. Towards that goal, it is important to understand the 3D reasoning capabilities of current models as well as identify the gaps between these models and humans. Therefore, we construct a new 3D visual understanding benchmark that covers fundamental 3D vision tasks in the Visual Question Answering (VQA) format. We evaluate state-of-the-art Vision-Language Models (VLMs), specialized models, and human subjects on it. Our results show that VLMs generally perform poorly, while the specialized models are accurate but not robust, failing under geometric perturbations. In contrast, human vision continues to be the most reliable 3D visual system. We further demonstrate that neural networks align more closely with human 3D vision mechanisms compared to classical computer vision methods, and Transformer-based networks such as ViT align more closely with human 3D vision mechanisms than CNNs. We hope our study will benefit the future development of foundation models for 3D vision.