Search

The inflation of an inner radial (or spherical) cavity in an amorphous solids confined in a disk (or a sphere), served as a fruitful case model for studying the effects of plastic deformations on the mechanical response. It was shown that when the field associated with Eshelby quadrupolar charges is non-uniform, the displacement field is riddled with dipole charges that screen elasticity, reminiscent of Debye monopoles screening in electrostatics. In this paper we look deeper into the screening phenomenon, taking into account the consequences of irreversibility that are associated with the breaking of Chiral symmetry. We consider the equations for the displacement field with the presence of ``Odd Dipole Screening", solve them analytically and compare with numerical simulations. Suggestions how to test the theory in experiments are provided.

Various noncollinear spin textures and magnetic phases have been predicted in twisted two-dimensional CrI$_3$ due to competing ferromagnetic (FM) and antiferromagnetic (AFM) interlayer exchange from moir\'e stacking - with potential spintronic applications even when the underlying material possesses a negligible Dzyaloshinskii-Moriya or dipole-dipole interaction. Recent measurements have shown evidence of coexisting FM and AFM layer order in small-twist-angle CrI$_3$ bilayers and double bilayers. Yet, the nature of the magnetic textures remains unresolved and possibilities for their manipulation and electrical readout are unexplored. Here, we use tunneling magnetoresistance to investigate the collective spin states of twisted double-bilayer CrI$_3$ under both out-of-plane and in-plane magnetic fields together with detailed micromagnetic simulations of domain dynamics based on magnetic circular dichroism. Our results capture hysteretic and anisotropic field evolutions of the magnetic states and we further uncover two distinct non-volatile spin textures (out-of-plane and in-plane domains) at $\approx$ 1{\deg} twist angle, with a different global tunneling resistance that can be switched by magnetic field., Comment: 18 pages, 5 figures

Vision Language Models (VLMs) have demonstrated remarkable performance in 2D vision and language tasks. However, their ability to reason about spatial arrangements remains limited. In this work, we introduce Spatial Region GPT (SpatialRGPT) to enhance VLMs' spatial perception and reasoning capabilities. SpatialRGPT advances VLMs' spatial understanding through two key innovations: (1) a data curation pipeline that enables effective learning of regional representation from 3D scene graphs, and (2) a flexible plugin module for integrating depth information into the visual encoder of existing VLMs. During inference, when provided with user-specified region proposals, SpatialRGPT can accurately perceive their relative directions and distances. Additionally, we propose SpatialRGBT-Bench, a benchmark with ground-truth 3D annotations encompassing indoor, outdoor, and simulated environments, for evaluating 3D spatial cognition in VLMs. Our results demonstrate that SpatialRGPT significantly enhances performance in spatial reasoning tasks, both with and without local region prompts. The model also exhibits strong generalization capabilities, effectively reasoning about complex spatial relations and functioning as a region-aware dense reward annotator for robotic tasks. Code, dataset, and benchmark will be released at https://www.anjiecheng.me/SpatialRGPT, Comment: Project Page: https://www.anjiecheng.me/SpatialRGPT

Novel view synthesis from a sparse set of input images is a challenging problem of great practical interest, especially when camera poses are absent or inaccurate. Direct optimization of camera poses and usage of estimated depths in neural radiance field algorithms usually do not produce good results because of the coupling between poses and depths, and inaccuracies in monocular depth estimation. In this paper, we leverage the recent 3D Gaussian splatting method to develop a novel construct-and-optimize method for sparse view synthesis without camera poses. Specifically, we construct a solution progressively by using monocular depth and projecting pixels back into the 3D world. During construction, we optimize the solution by detecting 2D correspondences between training views and the corresponding rendered images. We develop a unified differentiable pipeline for camera registration and adjustment of both camera poses and depths, followed by back-projection. We also introduce a novel notion of an expected surface in Gaussian splatting, which is critical to our optimization. These steps enable a coarse solution, which can then be low-pass filtered and refined using standard optimization methods. We demonstrate results on the Tanks and Temples and Static Hikes datasets with as few as three widely-spaced views, showing significantly better quality than competing methods, including those with approximate camera pose information. Moreover, our results improve with more views and outperform previous InstantNGP and Gaussian Splatting algorithms even when using half the dataset. Project page: https://raymondjiangkw.github.io/cogs.github.io/

3D hand-object interaction data is scarce due to the hardware constraints in scaling up the data collection process. In this paper, we propose HOIDiffusion for generating realistic and diverse 3D hand-object interaction data. Our model is a conditional diffusion model that takes both the 3D hand-object geometric structure and text description as inputs for image synthesis. This offers a more controllable and realistic synthesis as we can specify the structure and style inputs in a disentangled manner. HOIDiffusion is trained by leveraging a diffusion model pre-trained on large-scale natural images and a few 3D human demonstrations. Beyond controllable image synthesis, we adopt the generated 3D data for learning 6D object pose estimation and show its effectiveness in improving perception systems. Project page: https://mq-zhang1.github.io/HOIDiffusion, Comment: Project page: https://mq-zhang1.github.io/HOIDiffusion

An open problem in mobile manipulation is how to represent objects and scenes in a unified manner, so that robots can use it both for navigating in the environment and manipulating objects. The latter requires capturing intricate geometry while understanding fine-grained semantics, whereas the former involves capturing the complexity inherit to an expansive physical scale. In this work, we present GeFF (Generalizable Feature Fields), a scene-level generalizable neural feature field that acts as a unified representation for both navigation and manipulation that performs in real-time. To do so, we treat generative novel view synthesis as a pre-training task, and then align the resulting rich scene priors with natural language via CLIP feature distillation. We demonstrate the effectiveness of this approach by deploying GeFF on a quadrupedal robot equipped with a manipulator. We evaluate GeFF's ability to generalize to open-set objects as well as running time, when performing open-vocabulary mobile manipulation in dynamic scenes., Comment: Preprint. Project website is at: https://geff-b1.github.io/

We introduce a new RGB-D object dataset captured in the wild called WildRGB-D. Unlike most existing real-world object-centric datasets which only come with RGB capturing, the direct capture of the depth channel allows better 3D annotations and broader downstream applications. WildRGB-D comprises large-scale category-level RGB-D object videos, which are taken using an iPhone to go around the objects in 360 degrees. It contains around 8500 recorded objects and nearly 20000 RGB-D videos across 46 common object categories. These videos are taken with diverse cluttered backgrounds with three setups to cover as many real-world scenarios as possible: (i) a single object in one video; (ii) multiple objects in one video; and (iii) an object with a static hand in one video. The dataset is annotated with object masks, real-world scale camera poses, and reconstructed aggregated point clouds from RGBD videos. We benchmark four tasks with WildRGB-D including novel view synthesis, camera pose estimation, object 6d pose estimation, and object surface reconstruction. Our experiments show that the large-scale capture of RGB-D objects provides a large potential to advance 3D object learning. Our project page is https://wildrgbd.github.io/., Comment: Our project page: https://wildrgbd.github.io/

Despite Graph neural networks' significant performance gain over many classic techniques in various graph-related downstream tasks, their successes are restricted in shallow models due to over-smoothness and the difficulties of optimizations among many other issues. In this paper, to alleviate the over-smoothing issue, we propose a soft graph normalization method to preserve the diversities of node embeddings and prevent indiscrimination due to possible over-closeness. Combined with residual connections, we analyze the reason why the method can effectively capture the knowledge in both input graph structures and node features even with deep networks. Additionally, inspired by Curriculum Learning that learns easy examples before the hard ones, we propose a novel label-smoothing-based learning framework to enhance the optimization of deep GNNs, which iteratively smooths labels in an auxiliary graph and constructs many gradual non-smooth tasks for extracting increasingly complex knowledge and gradually discriminating nodes from coarse to fine. The method arguably reduces the risk of overfitting and generalizes better results. Finally, extensive experiments are carried out to demonstrate the effectiveness and potential of the proposed model and learning framework through comparison with twelve existing baselines including the state-of-the-art methods on twelve real-world node classification benchmarks., Comment: Accepted by AAAI 2024

While neural rendering has led to impressive advances in scene reconstruction and novel view synthesis, it relies heavily on accurately pre-computed camera poses. To relax this constraint, multiple efforts have been made to train Neural Radiance Fields (NeRFs) without pre-processed camera poses. However, the implicit representations of NeRFs provide extra challenges to optimize the 3D structure and camera poses at the same time. On the other hand, the recently proposed 3D Gaussian Splatting provides new opportunities given its explicit point cloud representations. This paper leverages both the explicit geometric representation and the continuity of the input video stream to perform novel view synthesis without any SfM preprocessing. We process the input frames in a sequential manner and progressively grow the 3D Gaussians set by taking one input frame at a time, without the need to pre-compute the camera poses. Our method significantly improves over previous approaches in view synthesis and camera pose estimation under large motion changes. Our project page is https://oasisyang.github.io/colmap-free-3dgs, Comment: Project Page: https://oasisyang.github.io/colmap-free-3dgs

Transition metal disulfides 4$H_{b}$-Ta(S, Se)$_{2}$ with natural heterostructure of 1${T}$- and 1${H}$-Ta(S, Se)$_{2}$ layers have became the focus of correlated materials their unique combinations of Mott physics and possible topological superconductivity. In this work, we study the upper critical fields $\mu_{0}H_{c2}$ of 4$H_{b}$-TaS$_{2}$ and 4$H_{b}$-TaS$_{1.99}$Se$_{0.01}$ single crystals systematically. Transport measurements up to 35 T show that both of ${ab}$-plane and ${c}$-axis upper critical fields ($\mu_{0}H_{c2,ab}$ and $\mu_{0}H_{c2,c}$) for 4$H_{b}$-TaS$_{2}$ and 4$H_{b}$-TaS$_{1.99}$Se$_{0.01}$ exhibit a linear temperature dependent behavior down to 0.3 K, suggesting the three-dimensional superconductivity with dominant orbital depairing mechanism in bulk 4$H_{b}$-Ta(S, Se)$_{2}$. However, the zero-temperature $\mu_{0}H_{c2,ab}$(0) for both crystals are far beyond the Pauli paramagnetic limit $\mu_{0}H{\rm_{P}}$. It could be explained by the effects of spin-momentum locking in 1$H$-Ta(S, Se)$_{2}$ layers with local inversion symmetry broken and the relatively weak intersublattice interaction between 1$H$ layers due to the existence of 1$T$ layers., Comment: 6 pages, 3 figures

High-fidelity 3D scene reconstruction has been substantially advanced by recent progress in neural fields. However, most existing methods train a separate network from scratch for each individual scene. This is not scalable, inefficient, and unable to yield good results given limited views. While learning-based multi-view stereo methods alleviate this issue to some extent, their multi-view setting makes it less flexible to scale up and to broad applications. Instead, we introduce training generalizable Neural Fields incorporating scene Priors (NFPs). The NFP network maps any single-view RGB-D image into signed distance and radiance values. A complete scene can be reconstructed by merging individual frames in the volumetric space WITHOUT a fusion module, which provides better flexibility. The scene priors can be trained on large-scale datasets, allowing for fast adaptation to the reconstruction of a new scene with fewer views. NFP not only demonstrates SOTA scene reconstruction performance and efficiency, but it also supports single-image novel-view synthesis, which is underexplored in neural fields. More qualitative results are available at: https://oasisyang.github.io/neural-prior, Comment: Project Page: https://oasisyang.github.io/neural-prior

We investigate the magnetotransport of topological Dirac semimetals (DSMs) by taking into account the Lifshitz transition of the Fermi arc surface states. We demonstrate that a bulk momentum-dependent gap term, which is usually neglected in study of the bulk energy-band topology, can cause the Lifshitz transition by developing an additional Dirac cone for the surface to prevent the Fermi arcs from connecting the bulk Dirac points. As a result, the Weyl orbits can be turned off by the surface Dirac cone without destroying the bulk Dirac points. In response to the surface Lifshitz transition, the Weyl-orbit mechanism for the 3D quantum Hall effect (QHE) in topological DSMs will break down. The resulting quantized Hall plateaus can be thickness-dependent, similar to the Weyl-orbit mechanism, but their widths and quantized values become irregular. Accordingly, we propose that apart from the bulk Weyl nodes and Fermi arcs, the surface Lifshitz transition is also crucial for realizing stable Weyl orbits and 3D QHE in real materials., Comment: 7

We present FastPoseGait, an open-source toolbox for pose-based gait recognition based on PyTorch. Our toolbox supports a set of cutting-edge pose-based gait recognition algorithms and a variety of related benchmarks. Unlike other pose-based projects that focus on a single algorithm, FastPoseGait integrates several state-of-the-art (SOTA) algorithms under a unified framework, incorporating both the latest advancements and best practices to ease the comparison of effectiveness and efficiency. In addition, to promote future research on pose-based gait recognition, we provide numerous pre-trained models and detailed benchmark results, which offer valuable insights and serve as a reference for further investigations. By leveraging the highly modular structure and diverse methods offered by FastPoseGait, researchers can quickly delve into pose-based gait recognition and promote development in the field. In this paper, we outline various features of this toolbox, aiming that our toolbox and benchmarks can further foster collaboration, facilitate reproducibility, and encourage the development of innovative algorithms for pose-based gait recognition. FastPoseGait is available at https://github.com//BNU-IVC/FastPoseGait and is actively maintained. We will continue updating this report as we add new features., Comment: 10 pages, 4 figures

Itinerant ferromagnetism at room temperature is a key ingredient for spin transport and manipulation. Here, we report the realization of nearly-room-temperature itinerant ferromagnetism in Co doped Fe5GeTe2 thin flakes. The ferromagnetic transition temperature TC (323 K - 337 K) is almost unchanged when thickness is down to 12 nm and is still about 284 K at 2 nm (bilayer thickness). Theoretical calculations further indicate that the ferromagnetism persists in monolayer Fe4CoGeTe2. In addition to the robust ferromagnetism down to the ultrathin limit, Fe4CoGeTe2 exhibits an unusual temperature- and thickness-dependent intrinsic anomalous Hall effect. We propose that it could be ascribed to the dependence of band structure on thickness that changes the Berry curvature near the Fermi energy level subtly. The nearly-room-temperature ferromagnetism and tunable anomalous Hall effect in atomically thin Fe4CoGeTe2 provide opportunities to understand the exotic transport properties of two-dimensional van der Waals magnetic materials and explore their potential applications in spintronics., Comment: 28 pages, 4 figures, 1 table

Effectively tuning magnetic state by using current is essential for novel spintronic devices. Magnetic van der Waals (vdW) materials have shown superior properties for the applications of magnetic information storage based on the efficient spin torque effect. However, for most of known vdW ferromagnets, the ferromagnetic transition temperatures lower than room temperature strongly impede their applications and the room-temperature vdW spintronic device with low energy consumption is still a long-sought goal. Here, we realize the highly efficient room-temperature nonvolatile magnetic switching by current in a single-material device based on vdW ferromagnet Fe3GaTe2. Moreover, the switching current density and power dissipation are about 300 and 60000 times smaller than conventional spin-orbit-torque devices of magnet/heavymetal heterostructures. These findings make an important progress on the applications of magnetic vdW materials in the fields of spintronics and magnetic information storage., Comment: 18 page2, 4 figures

Multivariate networks are commonly found in real-world data-driven applications. Uncovering and understanding the relations of interest in multivariate networks is not a trivial task. This paper presents a visual analytics workflow for studying multivariate networks to extract associations between different structural and semantic characteristics of the networks (e.g., what are the combinations of attributes largely relating to the density of a social network?). The workflow consists of a neural-network-based learning phase to classify the data based on the chosen input and output attributes, a dimensionality reduction and optimization phase to produce a simplified set of results for examination, and finally an interpreting phase conducted by the user through an interactive visualization interface. A key part of our design is a composite variable construction step that remodels nonlinear features obtained by neural networks into linear features that are intuitive to interpret. We demonstrate the capabilities of this workflow with multiple case studies on networks derived from social media usage and also evaluate the workflow with qualitative feedback from experts., Comment: To appear in IEEE Transactions on Visualization and Computer Graphics

Recent works on pose-based gait recognition have demonstrated the potential of using such simple information to achieve results comparable to silhouette-based methods. However, the generalization ability of pose-based methods on different datasets is undesirably inferior to that of silhouette-based ones, which has received little attention but hinders the application of these methods in real-world scenarios. To improve the generalization ability of pose-based methods across datasets, we propose a \textbf{G}eneralized \textbf{P}ose-based \textbf{Gait} recognition (\textbf{GPGait}) framework. First, a Human-Oriented Transformation (HOT) and a series of Human-Oriented Descriptors (HOD) are proposed to obtain a unified pose representation with discriminative multi-features. Then, given the slight variations in the unified representation after HOT and HOD, it becomes crucial for the network to extract local-global relationships between the keypoints. To this end, a Part-Aware Graph Convolutional Network (PAGCN) is proposed to enable efficient graph partition and local-global spatial feature extraction. Experiments on four public gait recognition datasets, CASIA-B, OUMVLP-Pose, Gait3D and GREW, show that our model demonstrates better and more stable cross-domain capabilities compared to existing skeleton-based methods, achieving comparable recognition results to silhouette-based ones. Code is available at https://github.com/BNU-IVC/FastPoseGait., Comment: ICCV Camera Ready

The quasi-one-dimensional material PdTeI exhibits unusual electronic transport properties at ambient pressure. Here, we systematically investigate both the structural and electronic responses of PdTeI to external pressure, through a combination of electrical transport, synchrotron x-ray diffraction (XRD), and Raman spectroscopy measurements. The charge density wave (CDW) order in PdTeI is fragile and the transition temperature TCDW decreases rapidly with the application of external pressure. The resistivity hump is indiscernible when the pressure is increased to 1 GPa. Upon further compression, zero resistance is established above 20 GPa, suggesting the occurrence of superconductivity. Combined XRD and Raman data evidence that the emergence of superconductivity is accompanied by a pressure-induced amorphization of PdTeI., Comment: 18 pages, 6 figures

While 6D object pose estimation has wide applications across computer vision and robotics, it remains far from being solved due to the lack of annotations. The problem becomes even more challenging when moving to category-level 6D pose, which requires generalization to unseen instances. Current approaches are restricted by leveraging annotations from simulation or collected from humans. In this paper, we overcome this barrier by introducing a self-supervised learning approach trained directly on large-scale real-world object videos for category-level 6D pose estimation in the wild. Our framework reconstructs the canonical 3D shape of an object category and learns dense correspondences between input images and the canonical shape via surface embedding. For training, we propose novel geometrical cycle-consistency losses which construct cycles across 2D-3D spaces, across different instances and different time steps. The learned correspondence can be applied for 6D pose estimation and other downstream tasks such as keypoint transfer. Surprisingly, our method, without any human annotations or simulators, can achieve on-par or even better performance than previous supervised or semi-supervised methods on in-the-wild images. Our project page is: https://kywind.github.io/self-pose ., Comment: Project page: https://kywind.github.io/self-pose

We propose a generalizable neural radiance fields - MonoNeRF, that can be trained on large-scale monocular videos of moving in static scenes without any ground-truth annotations of depth and camera poses. MonoNeRF follows an Autoencoder-based architecture, where the encoder estimates the monocular depth and the camera pose, and the decoder constructs a Multiplane NeRF representation based on the depth encoder feature, and renders the input frames with the estimated camera. The learning is supervised by the reconstruction error. Once the model is learned, it can be applied to multiple applications including depth estimation, camera pose estimation, and single-image novel view synthesis. More qualitative results are available at: https://oasisyang.github.io/mononerf ., Comment: ICML 2023 camera ready version. Project page: https://oasisyang.github.io/mononerf