Search

We present VR-Doh, a hands-on 3D modeling system designed for creating and manipulating elastoplastic objects in virtual reality (VR). The system employs the Material Point Method (MPM) for simulating realistic large deformations and incorporates optimized Gaussian Splatting for seamless rendering. With direct, hand-based interactions, users can naturally sculpt, deform, and edit objects interactively. To achieve real-time performance, we developed localized simulation techniques, optimized collision handling, and separated appearance and physical representations, ensuring smooth and responsive user interaction. The system supports both freeform creation and precise adjustments, catering to diverse modeling tasks. A user study involving novice and experienced users highlights the system's intuitive design, immersive feedback, and creative potential. Compared to traditional geometry-based modeling tools, our approach offers improved accessibility and natural interaction in specific contexts.

Incremental Potential Contact (IPC) is a widely used, robust, and accurate method for simulating complex frictional contact behaviors. However, achieving high efficiency remains a major challenge, particularly as material stiffness increases, which leads to slower Preconditioned Conjugate Gradient (PCG) convergence, even with the state-of-the-art preconditioners. In this paper, we propose a fully GPU-optimized IPC simulation framework capable of handling materials across a wide range of stiffnesses, delivering consistent high performance and scalability with up to 10x speedup over state-of-the-art GPU IPC methods. Our framework introduces three key innovations: 1) A novel connectivity-enhanced Multilevel Additive Schwarz (MAS) preconditioner on the GPU, designed to efficiently capture both stiff and soft elastodynamics and improve PCG convergence at a reduced preconditioning cost. 2) A C2-continuous cubic energy with an analytic eigensystem for strain limiting, enabling more parallel-friendly simulations of stiff membranes, such as cloth, without membrane locking. 3) For extremely stiff behaviors where elastic waves are barely visible, we employ affine body dynamics (ABD) with a hash-based multi-layer reduction strategy for fast Hessian assembly and efficient affine-deformable coupling. We conduct extensive performance analyses and benchmark studies to compare our framework against state-of-the-art methods and alternative design choices. Our system consistently delivers the fastest performance across soft, stiff, and hybrid simulation scenarios, even in cases with high resolution, large deformations, and high-speed impacts. Our framework will be fully open-sourced upon acceptance.

We propose a GPU-based iterative method for accelerated elastodynamic simulation with the log-barrier-based contact model. While Newton's method is a conventional choice for solving the interior-point system, the presence of ill-conditioned log barriers often necessitates a direct solution at each linearized substep and costs substantial storage and computational overhead. Moreover, constraint sets that vary in each iteration present additional challenges in algorithm convergence. Our method employs a novel barrier-augmented Lagrangian method to improve system conditioning and solver efficiency by adaptively updating an augmentation constraint sets. This enables the utilization of a scalable, inexact Newton-PCG solver with sparse GPU storage, eliminating the need for direct factorization. We further enhance PCG convergence speed with a domain-decomposed warm start strategy based on an eigenvalue spectrum approximated through our in-time assembly. Demonstrating significant scalability improvements, our method makes simulations previously impractical on 128 GB of CPU memory feasible with only 8 GB of GPU memory and orders-of-magnitude faster. Additionally, our method adeptly handles stiff problems, surpassing the capabilities of existing GPU-based interior-point methods. Our results, validated across various complex collision scenarios involving intricate geometries and large deformations, highlight the exceptional performance of our approach., Comment: 17 pages, 30 figures

As consumer Virtual Reality (VR) and Mixed Reality (MR) technologies gain momentum, there's a growing focus on the development of engagements with 3D virtual content. Unfortunately, traditional techniques for content creation, editing, and interaction within these virtual spaces are fraught with difficulties. They tend to be not only engineering-intensive but also require extensive expertise, which adds to the frustration and inefficiency in virtual object manipulation. Our proposed VR-GS system represents a leap forward in human-centered 3D content interaction, offering a seamless and intuitive user experience. By developing a physical dynamics-aware interactive Gaussian Splatting in a Virtual Reality setting, and constructing a highly efficient two-level embedding strategy alongside deformable body simulations, VR-GS ensures real-time execution with highly realistic dynamic responses. The components of our Virtual Reality system are designed for high efficiency and effectiveness, starting from detailed scene reconstruction and object segmentation, advancing through multi-view image in-painting, and extending to interactive physics-based editing. The system also incorporates real-time deformation embedding and dynamic shadow casting, ensuring a comprehensive and engaging virtual experience.Our project page is available at: https://yingjiang96.github.io/VR-GS/.

The Material Point Method (MPM) is a hybrid Eulerian Lagrangian simulation technique for solid mechanics with significant deformation. Structured background grids are commonly employed in the standard MPM, but they may give rise to several accuracy problems in handling complex geometries. When using (2D) unstructured triangular or (3D) tetrahedral background elements, however, significant challenges arise (\eg, cell-crossing error). Substantial numerical errors develop due to the inherent $\mathcal{C}^0$ continuity property of the interpolation function, which causes discontinuous gradients across element boundaries. Prior efforts in constructing $\mathcal{C}^1$ continuous interpolation functions have either not been adapted for unstructured grids or have only been applied to 2D triangular meshes. In this study, an Unstructured Moving Least Squares MPM (UMLS-MPM) is introduced to accommodate 2D and 3D simplex tessellation. The central idea is to incorporate a diminishing function into the sample weights of the MLS kernel, ensuring an analytically continuous velocity gradient estimation. Numerical analyses confirm the method's capability in mitigating cell crossing inaccuracies and realizing expected convergence.

We propose a hybrid neural network and physics framework for reduced-order modeling of elastoplasticity and fracture. State-of-the-art scientific computing models like the Material Point Method (MPM) faithfully simulate large-deformation elastoplasticity and fracture mechanics. However, their long runtime and large memory consumption render them unsuitable for applications constrained by computation time and memory usage, e.g., virtual reality. To overcome these barriers, we propose a reduced-order framework. Our key innovation is training a low-dimensional manifold for the Kirchhoff stress field via an implicit neural representation. This low-dimensional neural stress field (NSF) enables efficient evaluations of stress values and, correspondingly, internal forces at arbitrary spatial locations. In addition, we also train neural deformation and affine fields to build low-dimensional manifolds for the deformation and affine momentum fields. These neural stress, deformation, and affine fields share the same low-dimensional latent space, which uniquely embeds the high-dimensional simulation state. After training, we run new simulations by evolving in this single latent space, which drastically reduces the computation time and memory consumption. Our general continuum-mechanics-based reduced-order framework is applicable to any phenomena governed by the elastodynamics equation. To showcase the versatility of our framework, we simulate a wide range of material behaviors, including elastica, sand, metal, non-Newtonian fluids, fracture, contact, and collision. We demonstrate dimension reduction by up to 100,000X and time savings by up to 10X.

Recent advances in the simulation of frictionally contacting elastodynamics with the Incremental Potential Contact (IPC) model have enabled inversion and intersection-free simulation via the application of mollified barriers, filtered line-search, and optimization-based solvers for time integration. In its current formulation the IPC model is constructed via a discrete constraint model, replacing non-interpenetration constraints with barrier potentials on an already spatially discretized domain. However, while effective, this purely discrete formulation prohibits convergence under refinement. To enable a convergent IPC model we reformulate IPC potentials in the continuous setting and provide a first, convergent discretization thereof. We demonstrate and analyze the convergence behavior of this new model and discretization on a range of elastostatic and dynamic contact problems, and evaluate its accuracy on both analytical benchmarks and application-driven examples., Comment: 26 pages, 14 figures

In this work, we present a reconfigurable data glove design to capture different modes of human hand-object interactions, which are critical in training embodied artificial intelligence (AI) agents for fine manipulation tasks. To achieve various downstream tasks with distinct features, our reconfigurable data glove operates in three modes sharing a unified backbone design that reconstructs hand gestures in real time. In the tactile-sensing mode, the glove system aggregates manipulation force via customized force sensors made from a soft and thin piezoresistive material; this design minimizes interference during complex hand movements. The virtual reality (VR) mode enables real-time interaction in a physically plausible fashion: A caging-based approach is devised to determine stable grasps by detecting collision events. Leveraging a state-of-the-art finite element method (FEM), the simulation mode collects data on fine-grained 4D manipulation events comprising hand and object motions in 3D space and how the object's physical properties (e.g., stress and energy) change in accordance with manipulation over time. Notably, the glove system presented here is the first to use high-fidelity simulation to investigate the unobservable physical and causal factors behind manipulation actions. In a series of experiments, we characterize our data glove in terms of individual sensors and the overall system. More specifically, we evaluate the system's three modes by (i) recording hand gestures and associated forces, (ii) improving manipulation fluency in VR, and (iii) producing realistic simulation effects of various tool uses, respectively. Based on these three modes, our reconfigurable data glove collects and reconstructs fine-grained human grasp data in both physical and virtual environments, thereby opening up new avenues for the learning of manipulation skills for embodied AI agents., Comment: Paper accepted by Engineering

We present a robust and efficient method for simulating Lagrangian solid-fluid coupling based on a new operator splitting strategy. We use variational formulations to approximate fluid properties and solid-fluid interactions, and introduce a unified two-way coupling formulation for SPH fluids and FEM solids using interior point barrier-based frictional contact. We split the resulting optimization problem into a fluid phase and a solid-coupling phase using a novel time-splitting approach with augmented contact proxies, and propose efficient custom linear solvers. Our technique accounts for fluids interaction with nonlinear hyperelastic objects of different geometries and codimensions, while maintaining an algorithmically guaranteed non-penetrating criterion. Comprehensive benchmarks and experiments demonstrate the efficacy of our method., Comment: 14 pages, 13 figures

Recent breakthroughs in Vision-Language (V&L) joint research have achieved remarkable results in various text-driven tasks. High-quality Text-to-video (T2V), a task that has been long considered mission-impossible, was proven feasible with reasonably good results in latest works. However, the resulting videos often have undesired artifacts largely because the system is purely data-driven and agnostic to the physical laws. To tackle this issue and further push T2V towards high-level physical realism, we present an autonomous data generation technique and a dataset, which intend to narrow the gap with a large number of multi-modal, 3D Text-to-Video/Simulation (T2V/S) data. In the dataset, we provide high-resolution 3D physical simulations for both solids and fluids, along with textual descriptions of the physical phenomena. We take advantage of state-of-the-art physical simulation methods (i) Incremental Potential Contact (IPC) and (ii) Material Point Method (MPM) to simulate diverse scenarios, including elastic deformations, material fractures, collisions, turbulence, etc. Additionally, high-quality, multi-view rendering videos are supplied for the benefit of T2V, Neural Radiance Fields (NeRF), and other communities. This work is the first step towards fully automated Text-to-Video/Simulation (T2V/S). Live examples and subsequent work are at https://sites.google.com/view/tpa-net.

Learning the physical simulation on large-scale meshes with flat Graph Neural Networks (GNNs) and stacking Message Passings (MPs) is challenging due to the scaling complexity w.r.t. the number of nodes and over-smoothing. There has been growing interest in the community to introduce \textit{multi-scale} structures to GNNs for physical simulation. However, current state-of-the-art methods are limited by their reliance on the labor-intensive drawing of coarser meshes or building coarser levels based on spatial proximity, which can introduce wrong edges across geometry boundaries. Inspired by the bipartite graph determination, we propose a novel pooling strategy, \textit{bi-stride} to tackle the aforementioned limitations. Bi-stride pools nodes on every other frontier of the breadth-first search (BFS), without the need for the manual drawing of coarser meshes and avoiding the wrong edges by spatial proximity. Additionally, it enables a one-MP scheme per level and non-parametrized pooling and unpooling by interpolations, resembling U-Nets, which significantly reduces computational costs. Experiments show that the proposed framework, \textit{BSMS-GNN}, significantly outperforms existing methods in terms of both accuracy and computational efficiency in representative physical simulations., Comment: Updates summary: * update to the nwe ICML style

We introduce Midas, a robotics simulation framework based on the Incremental Potential Contact (IPC) model. Our simulator guarantees intersection-free, stable, and accurate resolution of frictional contact. We demonstrate the efficacy of our framework with experimental validations on high-precision tasks and through comparisons with Bullet physics. A reinforcement learning pipeline using Midas is also developed and tested to perform intersection-free peg-in-hole tasks.

Contact-aware topology optimization faces challenges in robustness, accuracy, and applicability to internal structural surfaces under self-contact. This work builds on the recently proposed barrier-based Incremental Potential Contact (IPC) model and presents a new self-contact-aware topology optimization framework. A combination of SIMP, adjoint sensitivity analysis, and the IPC frictional-contact model is investigated. Numerical examples for optimizing varying objective functions under contact are presented. The resulting algorithm proposed solves topology optimization for large deformation and complex frictionally contacting scenarios with accuracy and robustness.

Differentiable physics enables efficient gradient-based optimizations of neural network (NN) controllers. However, existing work typically only delivers NN controllers with limited capability and generalizability. We present a practical learning framework that outputs unified NN controllers capable of tasks with significantly improved complexity and diversity. To systematically improve training robustness and efficiency, we investigated a suite of improvements over the baseline approach, including periodic activation functions, and tailored loss functions. In addition, we find our adoption of batching and an Adam optimizer effective in training complex locomotion tasks. We evaluate our framework on differentiable mass-spring and material point method (MPM) simulations, with challenging locomotion tasks and multiple robot designs. Experiments show that our learning framework, based on differentiable physics, delivers better results than reinforcement learning and converges much faster. We demonstrate that users can interactively control soft robot locomotion and switch among multiple goals with specified velocity, height, and direction instructions using a unified NN controller trained in our system. Code is available at https://github.com/erizmr/Complex-locomotion-skill-learning-via-differentiable-physics.

Simulating stiff materials in applications where deformations are either not significant or can safely be ignored is a pivotal task across fields. Rigid body modeling has thus long remained a fundamental tool and is, by far, the most popular simulation strategy currently employed for modeling stiff solids. At the same time, numerical models of a rigid body continue to pose a number of known challenges and trade-offs including intersections, instabilities, inaccuracies, and/or slow performances that grow with contact-problem complexity. In this paper we revisit this problem and present ABD, a simple and highly effective affine body dynamics framework, which significantly improves state-of-the-art stiff simulations. We trace the challenges in the rigid-body IPC (incremental potential contact) method to the necessity of linearizing piecewise-rigid (SE(3)) trajectories and subsequent constraints. ABD instead relaxes the unnecessary (and unrealistic) constraint that each body's motion be exactly rigid with a stiff orthogonality potential, while preserving the rigid body model's key feature of a small coordinate representation. In doing so ABD replaces piecewise linearization with piecewise linear trajectories. This, in turn, combines the best from both parties: compact coordinates ensure small, sparse system solves, while piecewise-linear trajectories enable efficient and accurate constraint (contact and joint) evaluations. Beginning with this simple foundation, ABD preserves all guarantees of the underlying IPC model e.g., solution convergence, guaranteed non-intersection, and accurate frictional contact. Over a wide range and scale of simulation problems we demonstrate that ABD brings orders of magnitude performance gains (two- to three-order on the CPU and an order more utilizing the GPU, which is 10,000x speedups) over prior IPC-based methods with a similar or higher simulation quality.

We introduce a collection of benchmark problems in 2D and 3D (geometry description and boundary conditions), including simple cases with known analytic solution, classical experimental setups, and complex geometries with fabricated solutions for evaluation of numerical schemes for incompressible Navier-Stokes equations in laminar flow regime. We compare the performance of a representative selection of most broadly used algorithms for Navier-Stokes equations on this set of problems. Where applicable, we compare the most common spatial discretization choices (unstructured triangle/tetrahedral meshes and structured or semi-structured quadrilateral/hexahedral meshes). The study shows that while the type of spatial discretization used has a minor impact on the accuracy of the solutions, the choice of time integration method, spatial discretization order, and the choice of solving the coupled equations or reducing them to simpler subproblems have very different properties. Methods that are directly solving the original equations tend to be more accurate than splitting approaches for the same number of degrees of freedom, but numerical or computational difficulty arise when they are scaled to larger problem sizes. Low-order splitting methods are less accurate, but scale more easily to large problems, while higher-order splitting methods are accurate but require dense time discretizations to be stable. We release the description of the experiments and an implementation of our benchmark, which we believe will enable statistically significant comparisons with the state of the art as new approaches for solving the incompressible Navier-Stokes equations are introduced.

This paper introduces BFEMP, a new approach for monolithically coupling the Material Point Method (MPM) with the Finite Element Method (FEM) through barrier energy-based particle-mesh frictional contact using a variational time-stepping formulation. The fully implicit time integration of the coupled system is recast into a barrier-augmented unconstrained nonlinear optimization problem. A modified line-search Newton's method is adopted to strictly prevent material points from penetrating the FEM domain, ensuring convergence and feasibility regardless of the time step size or the mesh resolutions. The proposed coupling scheme also reduces to a new approach for imposing separable frictional kinematic boundaries for MPM when all nodal displacements in the FEM domain are prescribed with Dirichlet boundary conditions. Compared to standard implicit time integration, the extra algorithmic components associated with the contact treatment only depend on simple point-segment (or point-triangle in 3D) geometric queries which robustly handle arbitrary FEM mesh boundaries represented with codimension-1 simplices. Experiments and analyses are performed to demonstrate the robustness and accuracy of the proposed method., Comment: 28 pages, 17 figures

We present a barrier method for treating frictional contact on interfaces embedded in finite elements. The barrier treatment has several attractive features, including: (i) it does not introduce any additional degrees of freedom or iterative steps, (ii) it is free of inter-penetration, (iii) it avoids an ill-conditioned matrix system, and (iv) it allows one to control the solution accuracy directly. We derive the contact pressure from a smooth barrier energy function that is designed to satisfy the non-penetration constraint. Likewise, we make use of a smoothed friction law in which the stick-slip transition is described by a continuous function of the slip displacement. We discretize the formulation using the extended finite element method to embed interfaces inside elements, and devise an averaged surface integration scheme that effectively provides stable solutions without traction oscillations. Subsequently, we develop a way to tailor the parameters of the barrier method to embedded interfaces, such that the method can be used without parameter tuning. We verify and investigate the proposed method through numerical examples with varied levels of complexity. The numerical results demonstrate that the proposed method is remarkably robust for challenging frictional contact problems, while requiring low cost comparable to that of the penalty method.

We extend the incremental potential contact (IPC) model for contacting elastodynamics to resolve systems composed of codimensional DOFs in arbitrary combination. This enables a unified, interpenetration-free, robust, and stable simulation framework that couples codimension-0,1,2, and 3 geometries seamlessly with frictional contact. Extending IPC to thin structures poses new challenges in computing strain, modeling thickness and determining collisions. To address these challenges we propose three corresponding contributions. First, we introduce a C2 constitutive barrier model that directly enforces strain limiting as an energy potential while preserving rest state. This provides energetically-consistent strain limiting models (both isotropic and anisotropic) for cloth that enable strict satisfaction of strain-limit inequalities with direct coupling to both elastodynamics and contact via minimization of the incremental potential. Second, to capture the geometric thickness of codimensional domains we extend the IPC model to directly enforce distance offsets. Our treatment imposes a strict guarantee that mid-surfaces (resp. mid-lines) of shells (resp. rods) will not move closer than applied thickness values. This enables us to account for thickness in the contact behavior of codimensional structures and so robustly capture challenging contacting geometries; a number of which, to our knowledge, have not been simulated before. Third, codimensional models, especially with modeled thickness, mandate strict accuracy requirements that pose a severe challenge to all existing continuous collision detection (CCD) methods. To address these limitations we develop a new, efficient, simple-to-implement additive CCD (ACCD) method that applies conservative advancement to iteratively refine a lower bound for deforming primitives, converging to time of impact.

Unmanned aerial vehicles have been demonstrated successfully in a variety of tasks, including surveying and sampling tasks over large areas. These vehicles can take many forms. Quadrotors' agility and ability to hover makes them well suited for navigating potentially tight spaces, while fixed wing aircraft are capable of efficient flight over long distances. Hybrid aerial vehicles (HAVs) attempt to achieve both of these benefits by exhibiting multiple modes; however, morphing HAVs typically require extra actuators which add mass, reducing both agility and efficiency. We propose a morphing HAV with folding wings that exhibits both a quadrotor and a fixed wing mode without requiring any extra actuation. This is achieved by leveraging the motion of a bistable mechanism at the center of the aircraft to drive folding of the wing using only the existing motors and the inertia of the system. We optimize both the bistable mechanism and the folding wing using a topology optimization approach. The resulting mechanisms were fabricated on a 3D printer and attached to an existing quadrotor frame. Our prototype successfully transitions between both modes and our experiments demonstrate that the behavior of the fabricated prototype is consistent with that of the simulation., Comment: 7 pages, 10 figures

In this paper, a hybrid Lagrangian-Eulerian topology optimization (LETO) method is proposed to solve the elastic force equilibrium with the Material Point Method (MPM). LETO transfers density information from freely movable Lagrangian carrier particles to a fixed set of Eulerian quadrature points. This transfer is based on a smooth radial kernel involved in the compliance objective to avoid the artificial checkerboard pattern. The quadrature points act as MPM particles embedded in a lower-resolution grid and enable a sub-cell multi-density resolution of intricate structures with a reduced computational cost. A quadrature-level connectivity graph-based method is adopted to avoid the artificial checkerboard issues commonly existing in multi-resolution topology optimization methods. Numerical experiments are provided to demonstrate the efficacy of the proposed approach., Comment: 24 pages, 19 figures

We propose Hierarchical Optimization Time Integration (HOT) for efficient implicit time-stepping of the Material Point Method (MPM) irrespective of simulated materials and conditions. HOT is an MPM-specialized hierarchical optimization algorithm that solves nonlinear time step problems for large-scale MPM systems near the CFL-limit. HOT provides convergent simulations "out-of-the-box" across widely varying materials and computational resolutions without parameter tuning. As an implicit MPM time stepper accelerated by a custom-designed Galerkin multigrid wrapped in a quasi-Newton solver, HOT is both highly parallelizable and robustly convergent. As we show in our analysis, HOT maintains consistent and efficient performance even as we grow stiffness, increase deformation, and vary materials over a wide range of finite strain, elastodynamic and plastic examples. Through careful benchmark ablation studies, we compare the effectiveness of HOT against seemingly plausible alternative combinations of MPM with standard multigrid and other Newton-Krylov models. We show how these alternative designs result in severe issues and poor performance. In contrast, HOT outperforms the existing state-of-the-art, heavily optimized implicit MPM codes with an up to 10x performance speedup across a wide range of challenging benchmark test simulations.

Capturing the interaction between objects that have an extreme difference in Young s modulus or geometrical scale is a highly challenging topic for numerical simulation. One of the fundamental questions is how to build an accurate multi-scale method with optimal computational efficiency. In this work, we develop a material-point-spheropolygon discrete element method (MPM-SDEM). Our approach fully couples the material point method (MPM) and the spheropolygon discrete element method (SDEM) through the exchange of contact force information. It combines the advantage of MPM for accurately simulating elastoplastic continuum materials and the high efficiency of DEM for calculating the Newtonian dynamics of discrete near-rigid objects. The MPM-SDEM framework is demonstrated with an explicit time integration scheme. Its accuracy and efficiency are further analysed against the analytical and experimental data. Results demonstrate this method could accurately capture the contact force and momentum exchange between materials while maintaining favourable computational stability and efficiency. Our framework exhibits great potential in the analysis of multi-scale, multi-physics phenomena