Search

3D semantic occupancy prediction is an essential part of autonomous driving, focusing on capturing the geometric details of scenes. Off-road environments are rich in geometric information, therefore it is suitable for 3D semantic occupancy prediction tasks to reconstruct such scenes. However, most of researches concentrate on on-road environments, and few methods are designed for off-road 3D semantic occupancy prediction due to the lack of relevant datasets and benchmarks. In response to this gap, we introduce WildOcc, to our knowledge, the first benchmark to provide dense occupancy annotations for off-road 3D semantic occupancy prediction tasks. A ground truth generation pipeline is proposed in this paper, which employs a coarse-to-fine reconstruction to achieve a more realistic result. Moreover, we introduce a multi-modal 3D semantic occupancy prediction framework, which fuses spatio-temporal information from multi-frame images and point clouds at voxel level. In addition, a cross-modality distillation function is introduced, which transfers geometric knowledge from point clouds to image features.

We propose a novel method to construct ghost-free multiple scalar-tensor theories. The key idea is to use the geometric quantities of hypersurfaces defined by the scalar fields, rather than the covariant derivatives of scalar fields or spacetime curvature, to build the theory. This approach has proven effective in developing ghost-free scalar-tensor theories in the single-field case. When multiple scalar fields are present, each field specifies a foliation of spacelike hypersurfaces, on which we can define the normal vector, induced metric, extrinsic and intrinsic curvatures, as well as extrinsic (Lie) and intrinsic (spatial) derivatives, respectively. By employing these hypersurface geometric quantities as foundational elements, we construct the Lagrangian for interacting hypersurfaces that describes a multiple scalar-tensor theory. Given that temporal (Lie) and spatial derivatives are separated, it becomes relatively easier to control the order of time derivatives, thus helping to avoid ghost-like or unwanted degrees of freedom. In this work, we use bi-scalar-field theory as an example, focusing on polynomial-type Lagrangians. We construct monomials of hypersurface geometric quantities up to $d=3$, where $d$ denotes the number of derivatives in each monomial. Additionally, we present the correspondence between expressions in terms of hypersurface quantities and those in covariant bi-scalar-tensor theory. Through a cosmological perturbation analysis of a simple model, we demonstrate that the theory propagates two tensor and two scalar degrees of freedom at the linear order in perturbations, thereby remaining free from any extra degrees of freedom., Comment: 28 pages, 1 figure; v2, typos corrected

Research on autonomous driving in unstructured outdoor environments is less advanced than in structured urban settings due to challenges like environmental diversities and scene complexity. These environments-such as rural areas and rugged terrains-pose unique obstacles that are not common in structured urban areas. Despite these difficulties, autonomous driving in unstructured outdoor environments is crucial for applications in agriculture, mining, and military operations. Our survey reviews over 250 papers for autonomous driving in unstructured outdoor environments, covering offline mapping, pose estimation, environmental perception, path planning, end-to-end autonomous driving, datasets, and relevant challenges. We also discuss emerging trends and future research directions. This review aims to consolidate knowledge and encourage further research for autonomous driving in unstructured environments. To support ongoing work, we maintain an active repository with up-to-date literature and open-source projects at: https://github.com/chaytonmin/Survey-Autonomous-Driving-in-Unstructured-Environments., Comment: Survey paper; 38 pages

Cosmic muon imaging technology is increasingly being applied in various fields. However, simulating cosmic muons typically requires the rapid generation of a large number of muons and tracking their complex trajectories through intricate structures. This process is highly computationally demanding and consumes significant CPU time. To address these challenges, we introduce DeepMuon, an innovative deep learning model designed to efficiently and accurately generate cosmic muon distributions. In our approach, we employ the inverse Box-Cox transformation to reduce the kurtosis of the muon energy distribution, making it more statistically manageable for the model to learn. Additionally, we utilize the Sliced Wasserstein Distance (SWD) as a loss function to ensure precise simulation of the high-dimensional distributions of cosmic muons. We also demonstrate that DeepMuon can accurately learn muon distribution patterns from a limited set of data, enabling it to simulate real-world cosmic muon distributions as captured by detectors. Compared to traditional tools like CRY, DeepMuon significantly increases the speed of muon generation at sea level. Furthermore, we have developed a pipeline using DeepMuon that directly simulates muon distributions in underwater environments, dramatically accelerating simulations for underwater muon radiography and tomography. For more details on our open-source project, please visit https://github.com/wangab0/deepmuon.

We investigate the strong coupling problem in modified teleparallel gravity theories using the effective field theory (EFT) approach, demonstrating that it is possible to shift the emergence of new degrees of freedom (DoFs) to lower orders in perturbation theory. We first focus on the case of $f(T)$ gravity, and we show that in its conformally equivalent form the scalar perturbations are non-dynamical up to the cubic action. We then propose a simple modification of the theory, which lowers the appearance of new DoFs to cubic order, compared to the quartic order in standard $f(T)$ gravity. Our work opens a new avenue to address the issue of strong coupling in modified teleparallel gravity, and suggests a new classification scheme of these theories based on the perturbative order at which new DoFs appear., Comment: 20 pages

Contrastive image-text pre-trained models such as CLIP have shown remarkable adaptability to downstream tasks. However, they face challenges due to the high computational requirements of the Vision Transformer (ViT) backbone. Current strategies to boost ViT efficiency focus on pruning patch tokens but fall short in addressing the multimodal nature of CLIP and identifying the optimal subset of tokens for maximum performance. To address this, we propose greedy search methods to establish a "Golden Ranking" and introduce a lightweight predictor specifically trained to approximate this Ranking. To compensate for any performance degradation resulting from token pruning, we incorporate learnable visual tokens that aid in restoring and potentially enhancing the model's performance. Our work presents a comprehensive and systematic investigation of pruning tokens within the ViT backbone of CLIP models. Through our framework, we successfully reduced 40% of patch tokens in CLIP's ViT while only suffering a minimal average accuracy loss of 0.3 across seven datasets. Our study lays the groundwork for building more computationally efficient multimodal models without sacrificing their performance, addressing a key challenge in the application of advanced vision-language models.

In the digital era, table structure recognition technology is a critical tool for processing and analyzing large volumes of tabular data. Previous methods primarily focus on visual aspects of table structure recovery but often fail to effectively comprehend the textual semantics within tables, particularly for descriptive textual cells. In this paper, we introduce UniTabNet, a novel framework for table structure parsing based on the image-to-text model. UniTabNet employs a ``divide-and-conquer'' strategy, utilizing an image-to-text model to decouple table cells and integrating both physical and logical decoders to reconstruct the complete table structure. We further enhance our framework with the Vision Guider, which directs the model's focus towards pertinent areas, thereby boosting prediction accuracy. Additionally, we introduce the Language Guider to refine the model's capability to understand textual semantics in table images. Evaluated on prominent table structure datasets such as PubTabNet, PubTables1M, WTW, and iFLYTAB, UniTabNet achieves a new state-of-the-art performance, demonstrating the efficacy of our approach. The code will also be made publicly available.

The limited training samples for object detectors commonly result in low accuracy out-of-distribution (OOD) object detection. We have observed that feature vectors of the same class tend to cluster tightly in feature space, whereas those of different classes are more scattered. This insight motivates us to leverage feature similarity for OOD detection. Drawing on the concept of prototypes prevalent in few-shot learning, we introduce a novel network architecture, Proto-OOD, designed for this purpose. Proto-OOD enhances prototype representativeness through contrastive loss and identifies OOD data by assessing the similarity between input features and prototypes. It employs a negative embedding generator to create negative embedding, which are then used to train the similarity module. Proto-OOD achieves significantly lower FPR95 in MS-COCO dataset and higher mAP for Pascal VOC dataset, when utilizing Pascal VOC as ID dataset and MS-COCO as OOD dataset. Additionally, we identify limitations in existing evaluation metrics and propose an enhanced evaluation protocol., Comment: 14pages

In autonomous driving, 3D LiDAR plays a crucial role in understanding the vehicle's surroundings. However, the newly emerged, unannotated objects presents few-shot learning problem for semantic segmentation. This paper addresses the limitations of current few-shot semantic segmentation by exploiting the temporal continuity of LiDAR data. Employing a tracking model to generate pseudo-ground-truths from a sequence of LiDAR frames, our method significantly augments the dataset, enhancing the model's ability to learn on novel classes. However, this approach introduces a data imbalance biased to novel data that presents a new challenge of catastrophic forgetting. To mitigate this, we incorporate LoRA, a technique that reduces the number of trainable parameters, thereby preserving the model's performance on base classes while improving its adaptability to novel classes. This work represents a significant step forward in few-shot 3D LiDAR semantic segmentation for autonomous driving. Our code is available at https://github.com/junbao-zhou/Track-no-forgetting.

KV-Cache compression methods generally sample a KV-Cache of effectual tokens or quantize it into lower bits. However, these methods cannot exploit the redundancy of the hidden dimension of KV tensors. This paper investigates a unique hidden dimension approach called Palu, a novel KV-Cache compression framework that utilizes low-rank projection. Palu decomposes the linear layers into low-rank matrices, caches the smaller intermediate states, and reconstructs the full keys and values on the fly. To improve accuracy, compression rate, and efficiency, Palu further encompasses (1) a medium-grained low-rank decomposition scheme, (2) an efficient rank search algorithm, (3) a low-rank-aware quantization algorithm, and (4) matrix fusion with optimized GPU kernels. Our extensive experiments with popular LLMs show that Palu can compress KV-Cache by more than 91.25% while maintaining a significantly better accuracy (up to 1.19 lower perplexity) than state-of-the-art KV-Cache quantization methods at a similar or even higher memory usage. When compressing KV-Cache for 50%, Palu delivers up to 1.61x end-to-end speedup for the attention module. Our code is publicly available at https://github.com/shadowpa0327/Palu.

Swarm navigation in cluttered environments is a grand challenge in robotics. This work combines deep learning with first-principle physics through differentiable simulation to enable autonomous navigation of multiple aerial robots through complex environments at high speed. Our approach optimizes a neural network control policy directly by backpropagating loss gradients through the robot simulation using a simple point-mass physics model and a depth rendering engine. Despite this simplicity, our method excels in challenging tasks for both multi-agent and single-agent applications with zero-shot sim-to-real transfer. In multi-agent scenarios, our system demonstrates self-organized behavior, enabling autonomous coordination without communication or centralized planning - an achievement not seen in existing traditional or learning-based methods. In single-agent scenarios, our system achieves a 90% success rate in navigating through complex environments, significantly surpassing the 60% success rate of the previous state-of-the-art approach. Our system can operate without state estimation and adapt to dynamic obstacles. In real-world forest environments, it navigates at speeds up to 20 m/s, doubling the speed of previous imitation learning-based solutions. Notably, all these capabilities are deployed on a budget-friendly $21 computer, costing less than 5% of a GPU-equipped board used in existing systems. Video demonstrations are available at https://youtu.be/LKg9hJqc2cc.

Infrared imaging technology has gained significant attention for its reliable sensing ability in low visibility conditions, prompting many studies to convert the abundant RGB images to infrared images. However, most existing image translation methods treat infrared images as a stylistic variation, neglecting the underlying physical laws, which limits their practical application. To address these issues, we propose a Physics-Informed Diffusion (PID) model for translating RGB images to infrared images that adhere to physical laws. Our method leverages the iterative optimization of the diffusion model and incorporates strong physical constraints based on prior knowledge of infrared laws during training. This approach enhances the similarity between translated infrared images and the real infrared domain without increasing extra training parameters. Experimental results demonstrate that PID significantly outperforms existing state-of-the-art methods. Our code is available at https://github.com/fangyuanmao/PID.

Modern mainstream persistent key-value storage engines utilize Log-Structured Merge tree (LSM-tree) based designs, optimizing read/write performance by leveraging sequential disk I/O. However, the advent of SSDs, with their significant improvements in bandwidth and IOPS, shifts the bottleneck from I/O to CPU. The high compaction cost and large read/write amplification associated with LSM trees have become critical bottlenecks. In this paper, we introduce CompassDB, which utilizes a Two-tier Perfect Hash Table (TPH) design to significantly decrease read/write amplification and compaction costs. CompassDB utilizes a perfect hash algorithm for its in-memory index, resulting in an average index cost of about 6 bytes per key-value pair. This compact index reduces the lookup time complexity from $O(log N)$ to $O(1)$ and decreases the overall cost. Consequently, it allows for the storage of more key-value pairs for reads or provides additional memory for the memtable for writes. This results in substantial improvements in both throughput and latency. Our evaluation using the YCSB benchmark tool shows that CompassDB increases throughput by 2.5x to 4x compared to RocksDB, and by 5x to 17x compared to PebblesDB across six typical workloads. Additionally, CompassDB significantly reduces average and 99th percentile read/write latency, achieving a 50% to 85% reduction in comparison to RocksDB.

The competition between non-Hermitian skin effect and Anderson localization leads to various intriguing phenomena concerning spectrums and wavefunctions. Here, we study the linear response of disordered non-Hermitian systems, which is precisely described by the Green's function. We find that the average maximum value of matrix elements of Green's functions, which quantifies the maximum response against an external perturbation, exhibits different phases characterized by different scaling behaviors with respect to the system size. Whereas the exponential-growth phase is also seen in the translation-invariant systems, the algebraic-growth phase is unique to disordered non-Hermitian systems. We explain the findings using the large deviation theory, which provides analytical insights into the algebraic scaling factors of non-Hermitian disordered Green's functions. Furthermore, we show that these scaling behaviors can be observed in the steady states of disordered open quantum systems, offering a quantum-mechanical avenue for their experimental detection. Our work highlights an unexpected interplay between non-Hermitian skin effect and Anderson localization., Comment: 6+3 pages, 3+3 figures

Accurately identifying and organizing textual content is crucial for the automation of document processing in the field of form understanding. Existing datasets, such as FUNSD and XFUND, support entity classification and relationship prediction tasks but are typically limited to local and entity-level annotations. This limitation overlooks the hierarchically structured representation of documents, constraining comprehensive understanding of complex forms. To address this issue, we present the SRFUND, a hierarchically structured multi-task form understanding benchmark. SRFUND provides refined annotations on top of the original FUNSD and XFUND datasets, encompassing five tasks: (1) word to text-line merging, (2) text-line to entity merging, (3) entity category classification, (4) item table localization, and (5) entity-based full-document hierarchical structure recovery. We meticulously supplemented the original dataset with missing annotations at various levels of granularity and added detailed annotations for multi-item table regions within the forms. Additionally, we introduce global hierarchical structure dependencies for entity relation prediction tasks, surpassing traditional local key-value associations. The SRFUND dataset includes eight languages including English, Chinese, Japanese, German, French, Spanish, Italian, and Portuguese, making it a powerful tool for cross-lingual form understanding. Extensive experimental results demonstrate that the SRFUND dataset presents new challenges and significant opportunities in handling diverse layouts and global hierarchical structures of forms, thus providing deep insights into the field of form understanding. The original dataset and implementations of baseline methods are available at https://sprateam-ustc.github.io/SRFUND, Comment: NeurIPS 2024 Track on Datasets and Benchmarks under review

High-energy physics experiments rely heavily on precise measurements of energy and momentum, yet face significant challenges due to detector limitations, calibration errors, and the intrinsic nature of particle interactions. Traditional unfolding techniques have been employed to correct for these distortions, yet they often suffer from model dependency and stability issues. We present a novel method, SwdFold, which utilizes the principles of optimal transport to provide a robust, model-independent framework to estimate the probability density ratio for data unfolding. It not only unfold the toy experimental event by reweighted simulated data distributions closely with true distributions but also maintains the integrity of physical features across various observables. We can expect it can enable more reliable predictions and comprehensive analyses as a high precision reweighting and unfolding tool in high-energy physics.

We investigate the parity-violating scalar-tensor theory and pay special attention to terms that are free of the Ostrogradsky ghost in the unitary gauge, i.e., when the scalar field possesses a timelike gradient. We exhaustively identify the generally covariant scalar-tensor theory (GST) monomials with parity violation up to $d=4$, where $d$ is the total number of derivatives in the unitary gauge. According to the correspondence between GST terms and the spatially covariant gravity (SCG) terms in the unitary gauge, we also exhaustively identify the SCG monomials with parity violation up to $d=4$, where the Lie derivatives of the extrinsic curvature and the lapse function are necessarily introduced. We find a total of 9 independent parity-violating SCG monomials, of which 7 contain no higher-order Lie derivatives and are thus automatically free of ghosts, while 2 involve Lie derivatives of the extrinsic curvature and the lapse function and are thus potentially dangerous. By explicitly deriving their generally covariant correspondence, we obtain 7 independent scalar-tensor terms dubbed the ``Qi-Xiu'' Lagrangians, which are the most general parity-violating scalar-tensor theories that are ghost-free in the unitary gauge up to $d=4$. Our results include the existing theories in the literature, such as the Chern-Simons term and the chiral scalar-tensor theories, as special cases., Comment: 33 pages

The one-dimensional quantum breakdown model, which features spatially asymmetric fermionic interactions simulating the electrical breakdown phenomenon, exhibits an exponential U(1) symmetry and a variety of dynamical phases including many-body localization and quantum chaos with quantum scar states. We investigate the minimal quantum breakdown model with the minimal number of on-site fermion orbitals required for the interaction, and identify a large number of local conserved charges in the model. We then reveal a mapping between the minimal quantum breakdown model in certain charge sectors and a quantum link model which simulates the U(1) lattice gauge theory, and show that the local conserved charges map to the gauge symmetry generators. A special charge sector of the model further maps to the PXP model, which shows quantum many-body scars. This mapping unveils the rich dynamics in different Krylov subspaces characterized by different gauge configurations in the quantum breakdown model., Comment: 12 pages, 10 figures

We study the energy issue in critical collapse of a spherically symmetric scalar field. It is found that in critical collapse, the contribution from the material energy is greater than that from the gravitational energy. The quantity $m/r$ plays an important role in identifying the formation of apparent horizon in gravitational collapse, where $m$ is the Misner-Sharp mass and $r$ the areal radius. We observe that in critical collapse, the maximum value of $m/r$ fluctuates between $2/15$ and $4/15$. This denotes a large gap between critical collapse and black hole formation for which the criterion is $m/r=1/2$., Comment: 8 pages, 7 figures

Model uncertainty presents significant challenges in vibration suppression of multi-inertia systems, as these systems often rely on inaccurate nominal mathematical models due to system identification errors or unmodeled dynamics. An observer, such as an extended state observer (ESO), can estimate the discrepancy between the inaccurate nominal model and the true model, thus improving control performance via disturbance rejection. The conventional observer design is memoryless in the sense that once its estimated disturbance is obtained and sent to the controller, the datum is discarded. In this research, we propose a seamless integration of ESO and machine learning. On one hand, the machine learning model attempts to model the disturbance. With the assistance of prior information about the disturbance, the observer is expected to achieve faster convergence in disturbance estimation. On the other hand, machine learning benefits from an additional assurance layer provided by the ESO, as any imperfections in the machine learning model can be compensated for by the ESO. We validated the effectiveness of this novel learning-for-control paradigm through simulation and physical tests on two-inertial motion control systems used for vibration studies.

Images corrupted by rain streaks often lose vital frequency information for perception, and image deraining aims to solve this issue which relies on global and local degradation modeling. Recent studies have witnessed the effectiveness and efficiency of Mamba for perceiving global and local information based on its exploiting local correlation among patches, however, rarely attempts have been explored to extend it with frequency analysis for image deraining, limiting its ability to perceive global degradation that is relevant to frequency modeling (e.g. Fourier transform). In this paper, we propose FreqMamba, an effective and efficient paradigm that leverages the complementary between Mamba and frequency analysis for image deraining. The core of our method lies in extending Mamba with frequency analysis from two perspectives: extending it with frequency-band for exploiting frequency correlation, and connecting it with Fourier transform for global degradation modeling. Specifically, FreqMamba introduces complementary triple interaction structures including spatial Mamba, frequency band Mamba, and Fourier global modeling. Frequency band Mamba decomposes the image into sub-bands of different frequencies to allow 2D scanning from the frequency dimension. Furthermore, leveraging Mamba's unique data-dependent properties, we use rainy images at different scales to provide degradation priors to the network, thereby facilitating efficient training. Extensive experiments show that our method outperforms state-of-the-art methods both visually and quantitatively.

We propose a bosonic quantum breakdown Hubbard model, which generalizes the Bose-Hubbard model by adding an asymmetric breakdown interaction turning one boson into two between adjacent sites. We show that the ground state undergoes a first-order phase transition from Mott insulator to breakdown condensate as the breakdown interaction increases. When the normal hopping is zero, this model has a global exponential U(1) symmetry, which is spontaneously broken by the breakdown condensate, while the Mott-condensate transition remains first order. Surprisingly, the breakdown condensate with spontaneously broken exponential U(1) symmetry does not have a gapless Goldstone mode, which invalidates the Mermin-Wagner theorem and leads to stable spontaneous symmetry breaking in 1D. Moreover, we show that the model at non-equilibrium exhibits a dynamical transition from dielectric phase to breakdown phase with respect to the breakdown interaction. Our results show that quantum systems with exponential symmetry can exhibit novel unexpected properties., Comment: 8 pages, 6 figures, including the supplemental materials

Heavy-ion collisions provide a unique opportunity to explore nucleon-hyperon (N-Y) interactions through two-particle correlations. The $p-\Lambda$ and $d-\Lambda$ correlations shed light on both N-Y two-body and N-N-Y three-body interactions, which is crucial for understanding neutron star properties. We present the high precision measurement of $p-\Lambda$ and the first measurement of $d-\Lambda$ correlation with $\sqrt{s_{_{\rm NN}}}=$ 3 GeV Au+Au collisions at STAR. Using the Lednicky-Lyuboshitz formalism, we characterized emission source size, the scattering length ($f_0$), and the effective range ($d_0$) of $p-\Lambda$ and $d-\Lambda$ interactions. Using the $f_0$ and $d_0$ extracted from two spin states in $d-\Lambda$ correlation, the parameters from the doublet state indicate the hypertriton binding energy is consistent with the current average of world measurements., Comment: 4 pages, 3 figures, Quark Matter 2023 (QM23) proceedings

In this article, we investigate periodically driven open quantum systems within the framework of Floquet-Lindblad master equations. Specifically, we discuss Lindblad master equations in the presence of a coherent, time-periodic driving and establish their general spectral features. We also clarify the notions of transient and non-decaying solutions from this spectral perspective, and then prove that any physical system described by a Floquet-Lindblad equation must have at least one \textit{physical} non-equilibrium steady state (NESS), corresponding to an eigenoperator of the Floquet-Lindblad evolution superoperator $\mathcal{U}_F$ with unit eigenvalue. Since the Floquet-Lindblad formalism encapsulates the entire information regarding the NESS, it in principle enables us to obtain non-linear effects to all orders at once. The Floquet-Lindblad formalism thus provides a powerful tool for studying driven-dissipative solid-state systems, which we illustrate by deriving the nonlinear optical response of a simple two-band model of an insulating solid and comparing it with prior results established through Keldysh techniques., Comment: 19 pages, 2 figures

This paper presents a novel approach to network pruning, targeting block pruning in deep neural networks for edge computing environments. Our method diverges from traditional techniques that utilize proxy metrics, instead employing a direct block removal strategy to assess the impact on classification accuracy. This hands-on approach allows for an accurate evaluation of each block's importance. We conducted extensive experiments on CIFAR-10, CIFAR-100, and ImageNet datasets using ResNet architectures. Our results demonstrate the efficacy of our method, particularly on large-scale datasets like ImageNet with ResNet50, where it excelled in reducing model size while retaining high accuracy, even when pruning a significant portion of the network. The findings underscore our method's capability in maintaining an optimal balance between model size and performance, especially in resource-constrained edge computing scenarios.