Search

Long video understanding poses unique challenges due to their temporal complexity and low information density. Recent works address this task by sampling numerous frames or incorporating auxiliary tools using LLMs, both of which result in high computational costs. In this work, we introduce a curiosity-driven video agent with self-exploration capability, dubbed as VCA. Built upon VLMs, VCA autonomously navigates video segments and efficiently builds a comprehensive understanding of complex video sequences. Instead of directly sampling frames, VCA employs a tree-search structure to explore video segments and collect frames. Rather than relying on external feedback or reward, VCA leverages VLM's self-generated intrinsic reward to guide its exploration, enabling it to capture the most crucial information for reasoning. Experimental results on multiple long video benchmarks demonstrate our approach's superior effectiveness and efficiency.

Superparamagnetic tunnel junctions (sMTJs) are emerging as promising components for stochastic units in neuromorphic computing, owing to their tunable random switching behavior. Conventional MTJ control methods, such as spin-transfer torque (STT) and spin-orbit torque (SOT), often require substantial power. Here, we introduce the voltage-controlled exchange coupling (VCEC) mechanism, enabling switching between antiparallel and parallel states in sMTJs with an ultralow power consumption of only 40 nW, approximately two orders of magnitude lower than conventional STT-based sMTJs. This mechanism yields a sigmoid-shaped output response, making it ideally suited for neuromorphic computing applications. Furthermore, we validate the feasibility of integrating VCEC with the SOT current control, offering an additional dimension for magnetic state manipulation. This work marks the first practical demonstration of VCEC effect in sMTJs, highlighting its potential as a low-power control solution for probabilistic bits in advanced computing systems., Comment: 20 pages, 4 figures

Efficiently manipulating the magnetization of van der Waals ferromagnets has attracted considerable interest in developing room-temperature two-dimensional material-based memory and logic devices. Here, taking advantage of the unique properties of the van der Waals ferromagnet as well as promising characteristics of the orbital Hall effect, we demonstrate the room-temperature magnetization switching of van der Waals ferromagnet Fe3GaTe2 through the orbital torque generated by the orbital Hall material, Titanium (Ti). The switching current density is estimated to be around 1.6 x 10^6 A/cm^2, comparable to that achieved in Fe3GaTe2 using spin-orbit torque from spin Hall materials. The efficient magnetization switching arises from the combined effects of the large orbital Hall conductivity of Ti and the strong spin-orbit correlation of the Fe3GaTe2, as confirmed through theoretical calculations. Our findings advance the understanding of orbital torque switching and pave the way for exploring material-based orbitronic devices., Comment: 26 pages,4 figures, submitted

Deep learning (DL) methods have shown remarkable successes in medical image segmentation, often using large amounts of annotated data for model training. However, acquiring a large number of diverse labeled 3D medical image datasets is highly difficult and expensive. Recently, mask propagation DL methods were developed to reduce the annotation burden on 3D medical images. For example, Sli2Vol~\cite{yeung2021sli2vol} proposed a self-supervised framework (SSF) to learn correspondences by matching neighboring slices via slice reconstruction in the training stage; the learned correspondences were then used to propagate a labeled slice to other slices in the test stage. But, these methods are still prone to error accumulation due to the inter-slice propagation of reconstruction errors. Also, they do not handle discontinuities well, which can occur between consecutive slices in 3D images, as they emphasize exploiting object continuity. To address these challenges, in this work, we propose a new SSF, called \proposed, {for segmenting any anatomical structures in 3D medical images using only a single annotated slice per training and testing volume.} Specifically, in the training stage, we first propagate an annotated 2D slice of a training volume to the other slices, generating pseudo-labels (PLs). Then, we develop a novel Object Estimation Guided Correspondence Flow Network to learn reliable correspondences between consecutive slices and corresponding PLs in a self-supervised manner. In the test stage, such correspondences are utilized to propagate a single annotated slice to the other slices of a test volume. We demonstrate the effectiveness of our method on various medical image segmentation tasks with different datasets, showing better generalizability across different organs, modalities, and modals. Code is available at \url{https://github.com/adlsn/Sli2Volplus}

Reconstructing controllable Gaussian splats from monocular video is a challenging task due to its inherently insufficient constraints. Widely adopted approaches supervise complex interactions with additional masks and control signal annotations, limiting their real-world applications. In this paper, we propose an annotation guidance-free method, dubbed FreeGaussian, that mathematically derives dynamic Gaussian motion from optical flow and camera motion using novel dynamic Gaussian constraints. By establishing a connection between 2D flows and 3D Gaussian dynamic control, our method enables self-supervised optimization and continuity of dynamic Gaussian motions from flow priors. Furthermore, we introduce a 3D spherical vector controlling scheme, which represents the state with a 3D Gaussian trajectory, thereby eliminating the need for complex 1D control signal calculations and simplifying controllable Gaussian modeling. Quantitative and qualitative evaluations on extensive experiments demonstrate the state-of-the-art visual performance and control capability of our method. Project page: https://freegaussian.github.io.

Causal reasoning (CR) is a crucial aspect of intelligence, essential for problem-solving, decision-making, and understanding the world. While large language models (LLMs) can generate rationales for their outputs, their ability to reliably perform causal reasoning remains uncertain, often falling short in tasks requiring a deep understanding of causality. In this survey, we provide a comprehensive review of research aimed at enhancing LLMs for causal reasoning. We categorize existing methods based on the role of LLMs: either as reasoning engines or as helpers providing knowledge or data to traditional CR methods, followed by a detailed discussion of the methodologies in each category. We then evaluate the performance of LLMs on various causal reasoning tasks, providing key findings and in-depth analysis. Finally, we provide insights from current studies and highlight promising directions for future research. We aim for this work to serve as a comprehensive resource, fostering further advancements in causal reasoning with LLMs. Resources are available at https://github.com/chendl02/Awesome-LLM-causal-reasoning.

Collecting real-world manipulation trajectory data involving robotic arms is essential for developing general-purpose action policies in robotic manipulation, yet such data remains scarce. Existing methods face limitations such as high costs, labor intensity, hardware dependencies, and complex setup requirements involving SLAM algorithms. In this work, we introduce Fast-UMI, an interface-mediated manipulation system comprising two key components: a handheld device operated by humans for data collection and a robot-mounted device used during policy inference. Our approach employs a decoupled design compatible with a wide range of grippers while maintaining consistent observation perspectives, allowing models trained on handheld-collected data to be directly applied to real robots. By directly obtaining the end-effector pose using existing commercial hardware products, we eliminate the need for complex SLAM deployment and calibration, streamlining data processing. Fast-UMI provides supporting software tools for efficient robot learning data collection and conversion, facilitating rapid, plug-and-play functionality. This system offers an efficient and user-friendly tool for robotic learning data acquisition.

Magnetoelasticity plays a crucial role in numerous magnetic phenomena, including magnetocalorics, magnon excitation via acoustic waves, and ultrafast demagnetization/Einstein-de Haas effect. Despite a long-standing discussion on anisotropy-mediated magnetoelastic interactions of relativistic origin, including \textit{ab-initio} calculations, the exchange-mediated magnetoelastic parameters within an atomistic framework have only recently begun to be investigated. As a result, many of their behaviors and values for real materials remain poorly understood. Therefore, by using a proposed simple modification of the embedded cluster approach that reduces the computational complexity, we critically analyze the properties of exchange-mediated spin-lattice coupling parameters for elemental $3d$ ferromagnets (bcc Fe, fcc Ni, and fcc Co), comparing methods used for their extraction and relating their realistic values to symmetry considerations and orbitally-decomposed contributions. Additionally, we investigate the effects of noncollinearity (spin temperature) and applied pressure on these parameters. For Fe, we find that single-site rotations, associated with spin temperatures around $\sim100$ K, induce significant modifications, particularly in Dzyaloshinskii-Moriya-type couplings; in contrast, such interactions in Co and Ni remain almost configuration independent. Moreover, we demonstrate a notable change in the exchange-mediated magnetoelastic constants for Fe under isostatic compression. Finally, the conversion between atomistic, quantum-mechanically derived parameters and the phenomenological magnetoelastic theory is discussed, which can be an useful tool towards larger and more realistic dynamics simulations involving coupled subsystems.

GdRu$_2$Si$_2$ has recently drawn significant attention as a centrosymmetric magnet capable of hosting a short period skyrmion square lattice (SkL) in the absence of Dzyaloshinskii Moriya interaction (DMI). In this system, Gd atoms are arranged on a square lattice forming 2D layers separated by the Ru-Si network in the out-of-plane direction. In the low T regime, the ground state for zero/smaller external magnetic field ($\vec{B}_\perp$) along the out-of-plane direction is a single helical state, characterized by one modulation vector $\vec{Q}$ along one of the in-plane directions of the square lattice. For some critical range of higher $\vec{B}_\perp$, the helical state transforms into a SkL state that can be viewed as the overlap of two helical states defined with $\vec{Q}$ vectors in two in-plane directions, with the same magnitude of $\vec{Q}$ as for the single helical state. So far in the literature, importance has been given to this in-plane $\vec{Q}$ vector in understanding the magnetic phases of the system, considering the out-of-plane magnetic coupling to be weak, which therefore has been ignored. Our calculation of the Gd-Gd magnetic exchange interactions ($J_{ij}$) however shows the strongest $J_{ij}$ to occur between second neighbour Gd atoms along the [111] body-diagonal direction of the unit cell. This along with the body-centred tetragonal structure of the Gd sublattice points to the presence of a hitherto ignored modulation vector, $\vec{Q}_{[111]}$, along the [111] direction in the helical ground state. Atomistic Spin Dynamics (ASD) simulations show the importance of this interaction. This interlayer modulation vector $\vec{Q}_{[111]}$, along with the intralayer $\vec{Q}_{[100]}$, determines the total magnetic ordering of the system. Our data shows that the magnetic phases in GdRu$_2$Si$_2$ are far more complex than what has been previously discussed., Comment: 39 pages, 8 figures in the manuscript

The relationship between port-Hamiltonian and positive real descriptor systems is investigated. It is well-known that port-Hamiltonian systems are positive real, but the converse implication does not always hold. In [K. Cherifi, H. Gernandt, and D. Hinsen. The difference between port-Hamiltonian, passive and positive real descriptor systems. Math. Control Signals Systems, pages 1-32, 2023] sufficient conditions for the converse are presented. We refine these conditions and present for a completely controllable, completely observable and positive real descriptor system a necessary and sufficient condition as well as an explicit method to compute a port-Hamiltonian realization of a general positive real descriptor system.

3D Object Affordance Grounding aims to predict the functional regions on a 3D object and has laid the foundation for a wide range of applications in robotics. Recent advances tackle this problem via learning a mapping between 3D regions and a single human-object interaction image. However, the geometric structure of the 3D object and the object in the human-object interaction image are not always consistent, leading to poor generalization. To address this issue, we propose to learn generalizable invariant affordance knowledge from multiple human-object interaction images within the same affordance category. Specifically, we introduce the \textbf{M}ulti-\textbf{I}mage Guided Invariant-\textbf{F}eature-Aware 3D \textbf{A}ffordance \textbf{G}rounding (\textbf{MIFAG}) framework. It grounds 3D object affordance regions by identifying common interaction patterns across multiple human-object interaction images. First, the Invariant Affordance Knowledge Extraction Module (\textbf{IAM}) utilizes an iterative updating strategy to gradually extract aligned affordance knowledge from multiple images and integrate it into an affordance dictionary. Then, the Affordance Dictionary Adaptive Fusion Module (\textbf{ADM}) learns comprehensive point cloud representations that consider all affordance candidates in multiple images. Besides, the Multi-Image and Point Affordance (\textbf{MIPA}) benchmark is constructed and our method outperforms existing state-of-the-art methods on various experimental comparisons. Project page: \url{https://goxq.github.io/mifag}, Comment: Accepted by AAAI 2025

Current high-resolution vision-language models encode images as high-resolution image tokens and exhaustively take all these tokens to compute attention, which significantly increases the computational cost. To address this problem, we propose FlexAttention, a flexible attention mechanism for efficient high-resolution vision-language models. Specifically, a high-resolution image is encoded both as high-resolution tokens and low-resolution tokens, where only the low-resolution tokens and a few selected high-resolution tokens are utilized to calculate the attention map, which greatly shrinks the computational cost. The high-resolution tokens are selected via a high-resolution selection module which could retrieve tokens of relevant regions based on an input attention map. The selected high-resolution tokens are then concatenated to the low-resolution tokens and text tokens, and input to a hierarchical self-attention layer which produces an attention map that could be used for the next-step high-resolution token selection. The hierarchical self-attention process and high-resolution token selection process are performed iteratively for each attention layer. Experiments on multimodal benchmarks prove that our FlexAttention outperforms existing high-resolution VLMs (e.g., relatively ~9% in V* Bench, ~7% in TextVQA), while also significantly reducing the computational cost by nearly 40%., Comment: Accepted by ECCV 2024

Evaluating the performance of Simultaneous Localization and Mapping (SLAM) algorithms is essential for scientists and users of robotic systems alike. But there are a multitude of different permutations of possible options of hardware setups and algorithm configurations, as well as different datasets and algorithms, such that it was previously infeasible to thoroughly compare SLAM systems against the full state of the art. To solve that we present the SLAM Hive Benchmarking Suite, which is able to analyze SLAM algorithms in 1000's of mapping runs, through its utilization of container technology and deployment in the cloud. This paper presents the architecture and open source implementation of SLAM Hive and compares it to existing efforts on SLAM evaluation. We perform mapping runs with popular visual, RGBD and LiDAR based SLAM algorithms against commonly used datasets and show how SLAM Hive can be used to conveniently analyze the results against various aspects. Through this we envision that SLAM Hive can become an essential tool for proper comparisons and evaluations of SLAM algorithms and thus drive the scientific development in the research on SLAM. The open source software as well as a demo to show the live analysis of 1000's of mapping runs can be found on our SLAM Hive website., Comment: arXiv admin note: text overlap with arXiv:2303.11854

This paper presents the ShanghaiTech Mapping Robot, a state-of-the-art unmanned ground vehicle (UGV) designed for collecting comprehensive multi-sensor datasets to support research in robotics, Simultaneous Localization and Mapping (SLAM), computer vision, and autonomous driving. The robot is equipped with a wide array of sensors including RGB cameras, RGB-D cameras, event-based cameras, IR cameras, LiDARs, mmWave radars, IMUs, ultrasonic range finders, and a GNSS RTK receiver. The sensor suite is integrated onto a specially designed mechanical structure with a centralized power system and a synchronization mechanism to ensure spatial and temporal alignment of the sensor data. A 16-node on-board computing cluster handles sensor control, data collection, and storage. We describe the hardware and software architecture of the robot in detail and discuss the calibration procedures for the various sensors and investigate the interference for LiDAR and RGB-D sensors. The capabilities of the platform are demonstrated through an extensive outdoor dataset collected in a diverse campus environment. Experiments with two LiDAR-based and two RGB-based SLAM approaches showcase the potential of the dataset to support development and benchmarking for robotics. To facilitate research, we make the dataset publicly available along with the associated robot sensor calibration data: https://slam-hive.net/wiki/ShanghaiTech_Datasets, Comment: 19 pages, 27 figures. Submitted to IEEE Transactions on Robotics

This paper scales object-level reconstruction to complex scenes, advancing interactive scene reconstruction. We introduce two datasets, OmniSim and InterReal, featuring 28 scenes with multiple interactive objects. To tackle the challenge of inaccurate interactive motion recovery in complex scenes, we propose LiveScene, a scene-level language-embedded interactive radiance field that efficiently reconstructs and controls multiple objects. By decomposing the interactive scene into local deformable fields, LiveScene enables separate reconstruction of individual object motions, reducing memory consumption. Additionally, our interaction-aware language embedding localizes individual interactive objects, allowing for arbitrary control using natural language. Our approach demonstrates significant superiority in novel view synthesis, interactive scene control, and language grounding performance through extensive experiments. Project page: https://livescenes.github.io., Comment: Accepted at Neurips 2024

Toroidal classification and geometric duality in quantum spin systems is presented. Through our classification and duality, we reveal that various bipartite quantum features in magnon-systems can manifest equivalently in both bipartite ferromagnetic and antiferromagnetic materials, based upon the availability of relevant Hamiltonian parameters. Additionally, the results highlight the antiferromagnetic regime as an ultra-fast dual counterpart to the ferromagnetic regime, both exhibiting identical capabilities for quantum spintronics and technological applications. Concrete illustrations are provided, demonstrating how splitting and squeezing types of two-mode magnon quantum correlations can be realized across ferro- and antiferromagnetic regimes., Comment: 6 pages, 2 figures

While port-Hamiltonian descriptor systems are known to be stable and passive, they may not be asymptotically stable or strictly passive. Necessary and sufficient conditions are presented when these properties as well as the regularity and the index one property can be achieved via state feedback while preserving the port-Hamiltonian structure.

Magnetic skyrmions have garnered attention for their potential roles in spintronic applications, such as information carriers in computation, data storage, and nano-oscillators due to their small size, topological stability, and the requirement of small electric currents to manipulate them. Two key challenges in harnessing skyrmions are the stabilization requirement through a strong out-of-plane field, and the skyrmion Hall effect (SkHE). Here, we present a systematic model study of skyrmions in FM/AFM multi-layer structures by employing both atomistic Monte Carlo and atomistic spin dynamics simulations. We demonstrate that skyrmions stabilized by exchange bias have superior stability than field-stabilized skyrmions due to the formation of a magnetic imprint within the AFM layer. Additionally, stacking two skyrmion hosting FM layers between two antiferromagnetic (AFM) layers suppresses the SkHE and enables the transport of AFM-coupled skyrmions with high velocity in the order of a few Km/s. This proposed multi-layer configuration could serve as a pathway to overcome existing limitations in the development of skyrmion-based devices, and the insights obtained through this study contribute significantly to the broader understanding of topological spin textures in magnetic materials.

The impact of the local chemical environment on the Gilbert damping in the binary alloy Fe$_{100-x}$Co$_{x}$ is investigated, using computations based on density functional theory. By varying the alloy composition x as well as Fe/Co atom positions we reveal that the effective damping of the alloy is highly sensitive to the nearest neighbor environment, especially to the amount of Co and the average distance between Co-Co atoms at nearest neighbor sites. Both lead to a significant local increase (up to an order of magnitude) of the effective Gilbert damping, originating mainly from variations of the density of states at the Fermi energy. In a global perspective (i.e., making a configuration average for a real material), those differences in damping are masked by statistical averages. When low-temperature explicit atomistic dynamics simulations are performed, the impact of short-range disorder on local dynamics is observed to also alter the overall relaxation rate. Our results illustrate the possibility of local chemical engineering of the Gilbert damping, which may stimulate the study of new ways to tune and control materials aiming for spintronics applications.

YMn$_6$Sn$_6$ consists of two types of Mn-based kagome planes stacked along $c$-axis having a complex magnetic interactions. We report a spin reconstruction in YMn$_6$Sn$_6$ from ferromagnet (FM) into a combination of two incommensurate spin spirals (SSs) originating from two different type of Mn kagome planes driven by frustrated magnetic exchanges along the $c$-axis with inclusion of Hubbard U. The pitch angle and wave vector of the incommensurate SSs are $\sim$89.3$^{\circ}$ and $\sim$ (0 0 0.248) respectively which are in excellent agreement with experiment. We employed an effective model Hamiltonian constructed out of exchange interactions to capture experimentally observed non-equivalent nature of the two incommensurate SSs which also explain FM-SS crossover due to antiferromagntic spin exchange with correlation. We further report the existence of topological magnon with spin-orbit coupling in incommensurate SS phase of YMn$_6$Sn$_6$ by calculating the topological invariants and Berry curvature profile. The location of Dirac magnon in energy landscape at 73 meV matches with another experimental report. We demonstrate the accuracy of our results by highlighting experimental features in YMn$_6$Sn$_6$.

The structure preserving stabilization of (possibly non-regular) linear port-Hamiltonian descriptor (pHDAE) systems by output feedback is discussed. For general descriptor systems the characterization when there exist output feedbacks that lead to an asymptotically stable closed loop system is a very hard and partially an open problem. In contrast to this it is shown that for systems in pHDAE representation this problem can be completely solved. Necessary and sufficient conditions are presented that guarantee that there exist a proportional and/or derivative output feedback such that the resulting closed-loop port-Hamiltonian descriptor system is asymptotically stable. For this it is also necessary that the output feedback also makes the problem regular and of index at most one. A complete characterization when this is possible is presented as well.

The Landau-Lifshitz-Gilbert (LLG) and Landau-Lifshitz (LL) equations play an essential role for describing the dynamics of magnetization in solids. While a quantum analog of the LL dynamics has been proposed in [Phys. Rev. Lett. 110, 147201 (2013)], the corresponding quantum version of LLG remains unknown. Here, we propose such a quantum LLG equation that inherently conserves purity of the quantum state. We examine the quantum LLG dynamics of a dimer consisting of two interacting spin-1/2 particles. Our analysis reveals that, in the case of ferromagnetic coupling, the evolution of initially uncorrelated spins mirrors the classical LLG dynamics. However, in the antiferromagnetic scenario, we observe pronounced deviations from classical behavior, underscoring the unique dynamics of becoming a spinless state, which is non-locally correlated. Moreover, when considering spins that are initially entangled, our study uncovers an unusual form of revival-type quantum correlation dynamics, which differs significantly from what is typically seen in open quantum systems., Comment: Minor changes to main text; supplementary material added; journal reference added

The orbital Hall effect in light materials has attracted considerable attention for developing novel orbitronic devices. Here we investigate the orbital torque efficiency and demonstrate the switching of the perpendicularly magnetized materials through the orbital Hall material (OHM), i.e., Zirconium (Zr). The orbital torque efficiency of approximately 0.78 is achieved in the Zr OHM with the perpendicularly magnetized [Co/Pt]3 sample, which significantly surpasses that of the perpendicularly magnetized CoFeB/Gd/CoFeB sample (approximately 0.04). Such notable difference is attributed to the different spin-orbit correlation strength between the [Co/Pt]3 sample and the CoFeB/Gd/CoFeB sample, which has been confirmed through the theoretical calculations. Furthermore, the full magnetization switching of the [Co/Pt]3 sample with a switching current density of approximately 2.6x106 A/cm2 has been realized through Zr, which even outperforms that of the W spin Hall material. Our finding provides a guideline to understand orbital torque efficiency and paves the way to develop energy-efficient orbitronic devices., Comment: 21 pages, 4 figures, submitted

Despite extensive research on magnetic skyrmions and antiskyrmions, a significant challenge remains in crafting nontrivial high-order skyrmionic textures with varying, or even tailor-made, topologies. We address this challenge, by focusing on a construction pathway of skyrmionics metamaterial within a monolayer thin film and suggest several promising lattice-like, flakes-like, and cell-like skyrmionic metamaterials that are surprisingly stable. Central to our approach is the concept of 'simulated controlled assembly', in short, a protocol inspired by 'click chemistry' that allows for positioning topological magnetic structures where one likes, and then allowing for energy minimization to elucidate the stability. Utilizing high-throughput atomistic-spin-dynamic (ASD) simulations alongside state-of-the-art AI-driven tools, we have isolated skyrmions (topological charge Q=1), antiskyrmions (Q=-1), and skyrmionium (Q=0). These entities serve as foundational 'skyrmionic building blocks' to forming reported intricate textures. In this work, two key contributions are introduced to the field of skyrmionic systems. First, we present a novel method for integrating control assembly protocols for the stabilization and investigation of topological magnets, which marks a significant advancement in the ability to explore new skyrmionic textures. Second, we report on the discovery of skyrmionic metamaterials, which shows a plethora of complex topologies that are possible to investigate theoretically and experimentally.

In this paper, we introduce \textbf{GS-SLAM} that first utilizes 3D Gaussian representation in the Simultaneous Localization and Mapping (SLAM) system. It facilitates a better balance between efficiency and accuracy. Compared to recent SLAM methods employing neural implicit representations, our method utilizes a real-time differentiable splatting rendering pipeline that offers significant speedup to map optimization and RGB-D rendering. Specifically, we propose an adaptive expansion strategy that adds new or deletes noisy 3D Gaussians in order to efficiently reconstruct new observed scene geometry and improve the mapping of previously observed areas. This strategy is essential to extend 3D Gaussian representation to reconstruct the whole scene rather than synthesize a static object in existing methods. Moreover, in the pose tracking process, an effective coarse-to-fine technique is designed to select reliable 3D Gaussian representations to optimize camera pose, resulting in runtime reduction and robust estimation. Our method achieves competitive performance compared with existing state-of-the-art real-time methods on the Replica, TUM-RGBD datasets. Project page: https://gs-slam.github.io/., Comment: Accepted to CVPR 2024(highlight). Project Page: https://gs-slam.github.io/

Implicit neural SLAM has achieved remarkable progress recently. Nevertheless, existing methods face significant challenges in non-ideal scenarios, such as motion blur or lighting variation, which often leads to issues like convergence failures, localization drifts, and distorted mapping. To address these challenges, we propose EN-SLAM, the first event-RGBD implicit neural SLAM framework, which effectively leverages the high rate and high dynamic range advantages of event data for tracking and mapping. Specifically, EN-SLAM proposes a differentiable CRF (Camera Response Function) rendering technique to generate distinct RGB and event camera data via a shared radiance field, which is optimized by learning a unified implicit representation with the captured event and RGBD supervision. Moreover, based on the temporal difference property of events, we propose a temporal aggregating optimization strategy for the event joint tracking and global bundle adjustment, capitalizing on the consecutive difference constraints of events, significantly enhancing tracking accuracy and robustness. Finally, we construct the simulated dataset DEV-Indoors and real captured dataset DEV-Reals containing 6 scenes, 17 sequences with practical motion blur and lighting changes for evaluations. Experimental results show that our method outperforms the SOTA methods in both tracking ATE and mapping ACC with a real-time 17 FPS in various challenging environments. Project page: https://delinqu.github.io/EN-SLAM., Comment: Accept at CVPR 2024

Understanding the development and breakup of interfacial waves in a two-phase mixing layer between the gas and liquid streams is paramount to atomization. Due to the velocity difference between the two streams, the shear on the interface triggers a longitudinal instability, which develops to interfacial waves that propagate downstream. As the interfacial waves grow spatially, transverse modulations arise, turning the interfacial waves from quasi-2D to fully 3D. The inlet gas turbulence intensity has a strong impact on the interfacial instability. Therefore, parametric direct numerical simulations are performed in the present study to systematically investigate the effect of the inlet gas turbulence on the formation, development, and breakup of the interfacial waves. The open-source multiphase flow solver, PARIS, is used for the simulations and the mass-momentum consistent volume-of-fluid method is used to capture the sharp gas-liquid interfaces. Two computational domain widths are considered and the wide domain will allow a detailed study of the transverse development of the interfacial waves. The dominant frequency and spatial growth rate of the longitudinal instability are found to increase with the inlet gas turbulence intensity. The dominant transverse wavenumber, determined by the Rayleigh-Taylor instability, scales with the longitudinal frequency, so it also increases with the inlet gas turbulence intensity. The holes formed in the liquid sheet is important to the disintegration of the interfacial waves. The holes formation is influenced by the inlet gas turbulence. As a result, the sheet breakup dynamics and the statistics of the droplets formed also change accordingly.

We have investigated the dynamical magnetic properties of the V-based kagome stibnite compounds by combining the ab-initio calculated magnetic parameters of a spin Hamiltonian like inter-site exchange parameters, magnetocrystalline anisotropy and site projected magnetic moments, with full-fledged simulations of atomistic spin-dynamics. Our calculations reveal that in addition to a ferromagnetic order along the [001] direction, the system hosts a complex landscape of magnetic configurations comprised of commensurate and incommensurate spin-spirals along the [010] direction. The presence of such chiral magnetic textures may be the key to solve the mystery about the origin of the experimentally observed inherent breaking of the C6 rotational symmetry- and the time-reversal symmetry., Comment: Accepted In Physical Review B

The V-based kagome systems AV3Sb5 (A = Cs, Rb and K) are unique by virtue of the intricate interplay of non-trivial electronic structure, topology and intriguing fermiology, rendering them to be a playground of many mutually dependent exotic phases like charge-order and superconductivity. Despite numerous recent studies, the interconnection of magnetism and other complex collective phenomena in these systems has yet not arrived at any conclusion. Using first-principles tools, we demonstrate that their electronic structures, complex fermiologies and phonon dispersions are strongly influenced by the interplay of dynamic electron correlations, non-trivial spin-polarization and spin-orbit coupling. An investigation of the first-principles-derived inter-site magnetic exchanges with the complementary analysis of q-dependence of the electronic response functions and the electron-phonon coupling indicate that the system conforms as a frustrated spin-cluster, where the occurrence of the charge-order phase is intimately related to the mechanism of electron-phonon coupling, rather than the Fermi-surface nesting., Comment: Accepted in Physical Review Letters

A remarkable ability of human beings resides in compositional reasoning, i.e., the capacity to make "infinite use of finite means". However, current large vision-language foundation models (VLMs) fall short of such compositional abilities due to their "bag-of-words" behaviors and inability to construct words that correctly represent visual entities and the relations among the entities. To this end, we propose CoVLM, which can guide the LLM to explicitly compose visual entities and relationships among the text and dynamically communicate with the vision encoder and detection network to achieve vision-language communicative decoding. Specifically, we first devise a set of novel communication tokens for the LLM, for dynamic communication between the visual detection system and the language system. A communication token is generated by the LLM following a visual entity or a relation, to inform the detection network to propose regions that are relevant to the sentence generated so far. The proposed regions-of-interests (ROIs) are then fed back into the LLM for better language generation contingent on the relevant regions. The LLM is thus able to compose the visual entities and relationships through the communication tokens. The vision-to-language and language-to-vision communication are iteratively performed until the entire sentence is generated. Our framework seamlessly bridges the gap between visual perception and LLMs and outperforms previous VLMs by a large margin on compositional reasoning benchmarks (e.g., ~20% in HICO-DET mAP, ~14% in Cola top-1 accuracy, and ~3% on ARO top-1 accuracy). We also achieve state-of-the-art performances on traditional vision-language tasks such as referring expression comprehension and visual question answering.

We developed a method which performs the coupled adiabatic spin and lattice dynamics based on the tight-binding electronic structure model, where the intrinsic magnetic field and ionic forces are calculated from the converged self-consistent electronic structure at every time step. By doing so, this method allows us to explore limits where the physics described by a parameterized spin-lattice Hamiltonian is no longer accurate. We demonstrate how the lattice dynamics is strongly influenced by the underlying magnetic configuration, where disorder is able to induce significant lattice distortions. The presented method requires significantly less computational resources than ab initio methods, such as time-dependent density functional theory (TD-DFT). Compared to parameterized Hamiltonian-based methods, it also describes more accurately the dynamics of the coupled spin and lattice degrees of freedom, which becomes important outside of the regime of small lattice and spin fluctuations., Comment: 15 pages, 8 Figures