Search

We address a moving boundary problem that consists of a system of equations modeling an inviscid fluid interacting with a two-dimensional nonlinear Koiter plate at the boundary. We derive a priori estimates needed to prove the local-in-time existence of solutions. We use the Arbitrary Lagrange Euler (ALE) coordinates to fix the domain and obtain careful estimates for the nonlinear Koiter plate, ALE velocity, and pressure {without any viscoelastic smoothing}. For the nonlinear Koiter plate, higher order energy estimates are obtained, whereas estimates for the ALE pressure are obtained by setting up an elliptic problem. For the ALE velocity, the bounds are obtained through div-curl estimates by estimating the ALE vorticity. We then extend our results in two directions: (1) to include fractional Sobolev spaces and (2) to incorporate the normalized second fundamental form.

Image and video generative models that are pre-trained on Internet-scale data can greatly increase the generalization capacity of robot learning systems. These models can function as high-level planners, generating intermediate subgoals for low-level goal-conditioned policies to reach. However, the performance of these systems can be greatly bottlenecked by the interface between generative models and low-level controllers. For example, generative models may predict photorealistic yet physically infeasible frames that confuse low-level policies. Low-level policies may also be sensitive to subtle visual artifacts in generated goal images. This paper addresses these two facets of generalization, providing an interface to effectively "glue together" language-conditioned image or video prediction models with low-level goal-conditioned policies. Our method, Generative Hierarchical Imitation Learning-Glue (GHIL-Glue), filters out subgoals that do not lead to task progress and improves the robustness of goal-conditioned policies to generated subgoals with harmful visual artifacts. We find in extensive experiments in both simulated and real environments that GHIL-Glue achieves a 25% improvement across several hierarchical models that leverage generative subgoals, achieving a new state-of-the-art on the CALVIN simulation benchmark for policies using observations from a single RGB camera. GHIL-Glue also outperforms other generalist robot policies across 3/4 language-conditioned manipulation tasks testing zero-shot generalization in physical experiments., Comment: Code, model checkpoints and videos can be found at https://ghil-glue.github.io

This paper introduces an integer-valued generalized autoregressive conditional heteroskedasticity (INGARCH) model based on the novel geometric distribution and discusses some of its properties. The parameter estimation problem of the models are studied by conditional maximum likelihood and Bayesian approach using Hamiltonian Monte Carlo (HMC) algorithm. The results of the simulation studies and real data analysis affirm the good performance of the estimators and the model.

Building semantic 3D maps is valuable for searching for objects of interest in offices, warehouses, stores, and homes. We present a mapping system that incrementally builds a Language-Embedded Gaussian Splat (LEGS): a detailed 3D scene representation that encodes both appearance and semantics in a unified representation. LEGS is trained online as a robot traverses its environment to enable localization of open-vocabulary object queries. We evaluate LEGS on 4 room-scale scenes where we query for objects in the scene to assess how LEGS can capture semantic meaning. We compare LEGS to LERF and find that while both systems have comparable object query success rates, LEGS trains over 3.5x faster than LERF. Results suggest that a multi-camera setup and incremental bundle adjustment can boost visual reconstruction quality in constrained robot trajectories, and suggest LEGS can localize open-vocabulary and long-tail object queries with up to 66% accuracy.

Large policies pretrained on a combination of Internet-scale vision-language data and diverse robot demonstrations have the potential to change how we teach robots new skills: rather than training new behaviors from scratch, we can fine-tune such vision-language-action (VLA) models to obtain robust, generalizable policies for visuomotor control. Yet, widespread adoption of VLAs for robotics has been challenging as 1) existing VLAs are largely closed and inaccessible to the public, and 2) prior work fails to explore methods for efficiently fine-tuning VLAs for new tasks, a key component for adoption. Addressing these challenges, we introduce OpenVLA, a 7B-parameter open-source VLA trained on a diverse collection of 970k real-world robot demonstrations. OpenVLA builds on a Llama 2 language model combined with a visual encoder that fuses pretrained features from DINOv2 and SigLIP. As a product of the added data diversity and new model components, OpenVLA demonstrates strong results for generalist manipulation, outperforming closed models such as RT-2-X (55B) by 16.5% in absolute task success rate across 29 tasks and multiple robot embodiments, with 7x fewer parameters. We further show that we can effectively fine-tune OpenVLA for new settings, with especially strong generalization results in multi-task environments involving multiple objects and strong language grounding abilities, and outperform expressive from-scratch imitation learning methods such as Diffusion Policy by 20.4%. We also explore compute efficiency; as a separate contribution, we show that OpenVLA can be fine-tuned on consumer GPUs via modern low-rank adaptation methods and served efficiently via quantization without a hit to downstream success rate. Finally, we release model checkpoints, fine-tuning notebooks, and our PyTorch codebase with built-in support for training VLAs at scale on Open X-Embodiment datasets., Comment: Website: https://openvla.github.io/

Modern approaches to autonomous driving rely heavily on learned components trained with large amounts of human driving data via imitation learning. However, these methods require large amounts of expensive data collection and even then face challenges with safely handling long-tail scenarios and compounding errors over time. At the same time, pure Reinforcement Learning (RL) methods can fail to learn performant policies in sparse, constrained, and challenging-to-define reward settings such as autonomous driving. Both of these challenges make deploying purely cloned or pure RL policies in safety critical applications such as autonomous vehicles challenging. In this paper we propose Combining IMitation and Reinforcement Learning (CIMRL) approach - a safe reinforcement learning framework that enables training driving policies in simulation through leveraging imitative motion priors and safety constraints. CIMRL does not require extensive reward specification and improves on the closed loop behavior of pure cloning methods. By combining RL and imitation, we demonstrate that our method achieves state-of-the-art results in closed loop simulation and real world driving benchmarks.

The creation of large, diverse, high-quality robot manipulation datasets is an important stepping stone on the path toward more capable and robust robotic manipulation policies. However, creating such datasets is challenging: collecting robot manipulation data in diverse environments poses logistical and safety challenges and requires substantial investments in hardware and human labour. As a result, even the most general robot manipulation policies today are mostly trained on data collected in a small number of environments with limited scene and task diversity. In this work, we introduce DROID (Distributed Robot Interaction Dataset), a diverse robot manipulation dataset with 76k demonstration trajectories or 350 hours of interaction data, collected across 564 scenes and 84 tasks by 50 data collectors in North America, Asia, and Europe over the course of 12 months. We demonstrate that training with DROID leads to policies with higher performance and improved generalization ability. We open source the full dataset, policy learning code, and a detailed guide for reproducing our robot hardware setup., Comment: Project website: https://droid-dataset.github.io/

Utilizing the well-established Radial Tully-Fisher (RTF) relation observed in a `large' (843) sample of local galaxies, we report the maximum allowed variance in the Hubble parameter, $H_0$. We estimate the total intrinsic scatter in the magnitude of the RTF relation(s) implementing a cosmological model-independent cosmographic expansion. We find that the maximum allowed local variation in our baseline analysis, using 4 RTF relations in the galaxy sample is $\Delta H_0/H_0 \lesssim 3 \%$ at a $95\%$ C.L. significance. Which is implied form a constraint of $\Delta H_0/H_0 = 0.54^{+1.32}_{-1.37} \%$ estimated at $D_{\rm{L}}\sim 10\, [\rm{Mpc}]$. Using only one `best-constrained' radial bin we report a conservative $95\%$ C.L. limit of $\Delta H_0/H_0 \lesssim 4 \%$. Through our estimate of maximum variation, we propose a novel method to validate several late-time/local modifications put forth to alleviate the $H_0$ tension. We find that within the range of the current galaxy sample redshift distribution $10 \, [\rm{Mpc}] \le D_{\rm{L}} \le 140\, [\rm{Mpc}]$, it is highly unlikely to obtain a variation of $\Delta H_0/H_0 \sim 9\%$, necessary to alleviate the $H_0$-tension. However, we also elaborate on the possible alternative inferences when the innermost radial bin is included in the analysis. Alongside the primary analysis of fitting the individual RTF relations independently, we propose and perform a joint analysis of the RTF relations useful to create a pseudo-standardizable sample of galaxies. We also test for the spatial variation of $H_0$, finding that the current samples' galaxies distributed only in the southern hemisphere support the null hypothesis of isotropy, within the allowed noise levels., Comment: 14 pages, 13 figures, 2 tables

This article focuses on the coherent forecasting of the recently introduced novel geometric AR(1) (NoGeAR(1)) model - an INAR model based on inflated - parameter binomial thinning approach. Various techniques are available to achieve h - step ahead coherent forecasts of count time series, like median and mode forecasting. However, there needs to be more body of literature addressing coherent forecasting in the context of overdispersed count time series. Here, we study the forecasting distribution corresponding to NoGeAR(1) process using the Monte Carlo (MC) approximation method. Accordingly, several forecasting measures are employed in the simulation study to facilitate a thorough comparison of the forecasting capability of NoGeAR(1) with other models. The methodology is also demonstrated using real-life data, specifically the data on CW{\ss} TeXpert downloads and Barbados COVID-19 data.

This paper proposes a wavelet-based method for analysing periodic autoregressive moving average (PARMA) time series. Even though Fourier analysis provides an effective method for analysing periodic time series, it requires the estimation of a large number of Fourier parameters when the PARMA parameters do not vary smoothly. The wavelet-based analysis helps us to obtain a parsimonious model with a reduced number of parameters. We have illustrated this with simulated and actual data sets.

Visually-conditioned language models (VLMs) have seen growing adoption in applications such as visual dialogue, scene understanding, and robotic task planning; adoption that has fueled a wealth of new models such as LLaVa, InstructBLIP, and PaLI-3. Despite the volume of new releases, key design decisions around image preprocessing, architecture, and optimization are under-explored, making it challenging to understand what factors account for model performance $-$ a challenge further complicated by the lack of objective, consistent evaluations. To address these gaps, we first compile a suite of standardized evaluations spanning visual question answering, object localization, and challenge sets that probe properties such as hallucination; evaluations that provide fine-grained insight VLM capabilities. Second, we rigorously investigate VLMs along key design axes, including pretrained visual representations and training from base vs. instruct-tuned language models, amongst others. We couple our analysis with three resource contributions: (1) a unified framework for evaluating VLMs, (2) optimized, flexible training code, and (3) checkpoints for all models, including a family of VLMs at the 7-13B scale that strictly outperform InstructBLIP and LLaVa v1.5, the state-of-the-art in open VLMs., Comment: Published at ICML 2024. 22 pages, 11 figures. Training code and models: https://github.com/TRI-ML/prismatic-vlms. Evaluation code: https://github.com/TRI-ML/vlm-evaluation

Video foreground segmentation (VFS) is an important computer vision task wherein one aims to segment the objects under motion from the background. Most of the current methods are image-based, i.e., rely only on spatial cues while ignoring motion cues. Therefore, they tend to overfit the training data and don't generalize well to out-of-domain (OOD) distribution. To solve the above problem, prior works exploited several cues such as optical flow, background subtraction mask, etc. However, having a video data with annotations like optical flow is a challenging task. In this paper, we utilize the temporal information and the spatial cues from the video data to improve OOD performance. However, the challenge lies in how we model the temporal information given the video data in an interpretable way creates a very noticeable difference. We therefore devise a strategy that integrates the temporal context of the video in the development of VFS. Our approach give rise to deep learning architectures, namely MUSTAN1 and MUSTAN2 and they are based on the idea of multi-scale temporal context as an attention, i.e., aids our models to learn better representations that are beneficial for VFS. Further, we introduce a new video dataset, namely Indoor Surveillance Dataset (ISD) for VFS. It has multiple annotations on a frame level such as foreground binary mask, depth map, and instance semantic annotations. Therefore, ISD can benefit other computer vision tasks. We validate the efficacy of our architectures and compare the performance with baselines. We demonstrate that proposed methods significantly outperform the benchmark methods on OOD. In addition, the performance of MUSTAN2 is significantly improved on certain video categories on OOD data due to ISD., Comment: 10 pages, 8 figures

We propose and perform a joint analysis of the two different mass estimates of galaxy clusters, namely the hydrostatic and caustic techniques. Firstly, we show comprehensively that the mass bias between these two techniques can be possibly alleviated when cluster-specific assumptions constrained using the hydrostatic technique are utilized within the caustic technique. While at face value this demotes the caustic technique from a completely independent method, this allows one to further tighten the constraints on the cluster mass and subsequently, allow us to test modifications to gravity. Implementing the aforementioned formalism for two well-observed massive galaxy clusters, A2029 and A2142, we highlight the proof of concept. In the current implementation, we use this method to constrain the Chameleon screening and Vainshtein screening. As anticipated, we show that the joint analysis can help improve the constraints on these modified gravity scenarios., Comment: 17 pages, 9 figures. Agrees with the accepted version, references added

Audio Question Answering (AQA) constitutes a pivotal task in which machines analyze both audio signals and natural language questions to produce precise natural language answers. The significance of possessing high-quality, diverse, and extensive AQA datasets cannot be overstated when aiming for the precision of an AQA system. While there has been notable focus on developing accurate and efficient AQA models, the creation of high-quality, diverse, and extensive datasets for the specific task at hand has not garnered considerable attention. To address this challenge, this work makes several contributions. We introduce a scalable AQA data generation pipeline, denoted as the AQUALLM framework, which relies on Large Language Models (LLMs). This framework utilizes existing audio-caption annotations and incorporates state-of-the-art LLMs to generate expansive, high-quality AQA datasets. Additionally, we present three extensive and high-quality benchmark datasets for AQA, contributing significantly to the progression of AQA research. AQA models trained on the proposed datasets set superior benchmarks compared to the existing state-of-the-art. Moreover, models trained on our datasets demonstrate enhanced generalizability when compared to models trained using human-annotated AQA data. Code and datasets will be accessible on GitHub~\footnote{\url{https://github.com/swarupbehera/AQUALLM}}.

Large, high-capacity models trained on diverse datasets have shown remarkable successes on efficiently tackling downstream applications. In domains from NLP to Computer Vision, this has led to a consolidation of pretrained models, with general pretrained backbones serving as a starting point for many applications. Can such a consolidation happen in robotics? Conventionally, robotic learning methods train a separate model for every application, every robot, and even every environment. Can we instead train generalist X-robot policy that can be adapted efficiently to new robots, tasks, and environments? In this paper, we provide datasets in standardized data formats and models to make it possible to explore this possibility in the context of robotic manipulation, alongside experimental results that provide an example of effective X-robot policies. We assemble a dataset from 22 different robots collected through a collaboration between 21 institutions, demonstrating 527 skills (160266 tasks). We show that a high-capacity model trained on this data, which we call RT-X, exhibits positive transfer and improves the capabilities of multiple robots by leveraging experience from other platforms. More details can be found on the project website https://robotics-transformer-x.github.io., Comment: Project website: https://robotics-transformer-x.github.io

[abridged] We propose a model of the Universe (dubbed $\eta$CDM) featuring a stochastic evolution of the cosmological quantities, that is meant to render small deviations from homogeneity/isotropy on scales of $30-50\, h^{-1}$ Mpc at late cosmic times, associated to the emergence of the cosmic web. Specifically, we prescribe that the behavior of the matter/radiation energy densities in different patches of the Universe with such a size can be effectively described by a stochastic version of the mass-energy evolution equation. The latter includes an appropriate noise term that statistically accounts for local fluctuations due to inhomogeneities, anisotropic stresses and matter flows. The evolution of the different patches as a function of cosmic time is rendered via the diverse realizations of the noise term; meanwhile, at any given cosmic time, sampling the ensemble of patches will originate a nontrivial spatial distribution of the cosmological quantities. The overall behavior of the Universe will be obtained by averaging over the patch ensemble. We assume a physically reasonable parameterization of the noise term, gauging it against a wealth of cosmological datasets. We find that, with respect to standard $\Lambda$CDM, the ensemble-averaged cosmic dynamics in the $\eta$CDM model is substantially altered in three main respects: (i) an accelerated expansion is enforced at late cosmic times without the need for any additional exotic component (e.g., dark energy); (ii) the spatial curvature can stay small even in a low-density Universe; (iii) matter can acquire an effective negative pressure at late times. We provide predictions for the variance of the cosmological quantities among different patches of the Universe at late cosmic times. Finally, we show that in $\eta$CDM the Hubble-tension is solved, and the cosmic coincidence problem is relieved without invoking the anthropic principle., Comment: 28 pages, 7 figures, typos corrected. Accepted by ApJ

We present a multiscale theoretical framework to investigate the interplay between diffusion and finite lattice deformation in phase transformation materials. In this framework, we use the Cauchy-Born Rule and the Principle of Virtual Power to derive a thermodynamically consistent theory coupling the diffusion of a guest species (Cahn-Hilliard type) with the finite deformation of host lattices (nonlinear gradient elasticity). We adapt this theory to intercalation materials--specifically Li$_{1-2}$Mn$_2$O$_4$--to investigate the delicate interplay between Li-diffusion and the cubic-to-tetragonal deformation of lattices. Our computations reveal fundamental insights into the microstructural evolution pathways under dynamic discharge conditions, and provide quantitative insights into the nucleation and growth of twinned microstructures during intercalation. Additionally, our results identify regions of stress concentrations (e.g., at phase boundaries, particle surfaces) that arise from lattice misfit and accumulate in the electrode with repeated cycling. These findings suggest a potential mechanism for structural decay in Li$_2$Mn$_2$O$_4$. More generally, we establish a theoretical framework that can be used to investigate microstructural evolution pathways, across multiple length scales, in first-order phase transformation materials., Comment: 41 pages, 11 figures

In this paper, we present a benchmarking study of vision-based grasp synthesis algorithms, each with distinct approaches, and provide a comparative analysis of their performance under different experimental conditions. In particular, we compare two machine-learning-based and two analytical algorithms to determine their strengths and weaknesses in different scenarios. In addition, we provide an open-source benchmarking tool developed from state-of-the-art benchmarking procedures and protocols to systematically evaluate different grasp synthesis algorithms. Our findings offer insights into the performance of the evaluated algorithms, which can aid in selecting the most appropriate algorithm for different scenarios., Comment: Submitted to International Symposium on Experimental Robotics (ISER) 2023

[abridged] Recently, in Benetti et al. (Astrophys. J. 2023, 949, 65), we suggested that the dark matter (DM) component in galaxies may originate fractional gravity. In such a framework, the DM component exists, but the gravitational potential associated to its density distribution is determined by a modified Poisson equation including fractional derivatives, which are meant to describe nonlocal effects. In Benetti et al., we showed that fractional gravity worked very well for reproducing the kinematics of disk-dominated galaxies, especially dwarfs; there is also preliminary evidence that the strength of fractional effects tends to weaken toward more massive systems. Here, we aim to test fractional gravity in galaxy clusters, with a twofold aim: (i) perform an independent sanity check that it can accurately describe such large and massive structures; (ii) derive a clear-cut trend for its strength in systems with different DM masses. To this purpose, we forward model the density and pressure distributions of the intracluster medium (ICM), working out the hydrostatic equilibrium equation in fractional gravity. Then, we perform a Bayesian analysis of the X-COP galaxy cluster sample and infer constraints on the fractional gravity parameters, for individual clusters as well as stacked clusters. We find that fractional gravity performs remarkably well in modeling the ICM profiles for the X-COP sample. We also confirm the weakening of the fractional gravity effects toward more massive systems and derive the overall scaling of the fractional gravity parameters from dwarf galaxies to massive clusters, spanning six orders of magnitude in DM mass. Such an overall trend implies that fractional gravity can substantially alleviate the small-scale issues of the standard DM paradigm, while remaining successful on large cosmological scales., Comment: 17 pages, 8 figures, Accepted by Universe