Search

Sim2Real transfer, particularly for manipulation policies relying on RGB images, remains a critical challenge in robotics due to the significant domain shift between synthetic and real-world visual data. In this paper, we propose SplatSim, a novel framework that leverages Gaussian Splatting as the primary rendering primitive to reduce the Sim2Real gap for RGB-based manipulation policies. By replacing traditional mesh representations with Gaussian Splats in simulators, SplatSim produces highly photorealistic synthetic data while maintaining the scalability and cost-efficiency of simulation. We demonstrate the effectiveness of our framework by training manipulation policies within SplatSim}and deploying them in the real world in a zero-shot manner, achieving an average success rate of 86.25%, compared to 97.5% for policies trained on real-world data.

This note studies projective planes having a collineation group fixing a flag $(\infty,L_\infty )$ and transitive on the flags $(w,W )$ with $w\notin L_\infty $ and $\infty\notin W$.

In this study, high pressure synchrotron powder X-ray diffraction is used to investigate the compression of two high pressure polymorphs of \ce{CrSb2}. The first is the \ce{CuAl2}-type polymorph with an eight-fold coordinated Cr, which can be quenched to ambient conditions from high-pressure high-temperature conditions. The second is the recently discovered \ce{MoP2}-type polymorph, which is induced by compression at room temperature, with a seven-fold coordinated Cr. Here, the assigned structure is unambiguously confirmed by solving it from single-crystal X-ray diffraction. Furthermore, the electrical properties of the \ce{MoP2}-type polymorph were investigated theoretically and the resistance calculations under pressure were accompanied by resistance measurements under high pressure on a single crystal of \ce{CrSb2}. The calculated electronic band structure for the \ce{MoP2}-type phase is discussed and we show that the polymorph is semimetallic and possesses type-I Weyl points. No further phase transitions were observed for the \ce{CuAl2}-type structure up to 50 GPa and 40 GPa for the \ce{MoP2}-type structure. Even though the \ce{CuAl2}-phase has the highest coordination number of Cr, it was found to be less compressible than the \ce{MoP2}-phase having a seven-fold coordinated Cr, which was attributed to the longer Cr-Sb distance in the \ce{CuAl2}-type phase. The discovery of a type-I Weyl semimetallic phase in \ce{CrSb2} opens up for discovering other Weyl semimetals in the transition metal di-pnictides under high pressure., Comment: 20 pages, 4 tables, 6 figures

We propose MMLU-SR, a novel dataset designed to measure the true comprehension abilities of Large Language Models (LLMs) by challenging their performance in question-answering tasks with modified terms. We reasoned that an agent that ``truly'' understands a concept can still evaluate it when key terms are replaced by suitably defined alternate terms, and sought to differentiate such comprehension from mere text replacement. In our study, we modified standardized test questions by replacing a key term with a dummy word along with its definition. The key term could be in the context of questions, answers, or both questions and answers. Notwithstanding the high scores achieved by recent popular LLMs on the MMLU leaderboard, we found a substantial reduction in model performance after such replacement, suggesting poor comprehension. This new benchmark provides a rigorous benchmark for testing true model comprehension, and poses a challenge to the broader scientific community.

It is commonly believed that the dissipative properties of superdense matter play a negligible role in modeling gravitational waveforms from neutron star inspirals. This study aims to investigate whether this presumption holds true for the often neglected dissipative process associated with particle diffusion in superconducting neutron stars. As we demonstrate, diffusion effects can significantly impact the phase of the gravitational wave from the inspiral, manifesting at a magnitude of a few tens of milliradians at large orbit separations, equivalent to orbital frequencies of a few hertz. We also find that dissipation resulting from particle diffusion might increase the neutron star's temperature to approximately $10^7\rm K$ during the inspiral., Comment: 29 pages, 11 figures, 1 table, accepted to Phys. Rev. D (selected as Editors' Suggestion)

We consider a $d$-dimensional correlated percolation problem of sites {\em not} visited by a random walk on a hypercubic lattice $L^d$ for $d=3$, 4 and 5. The length of the random walk is ${\cal N}=uL^d$. Close to the critical value $u=u_c$, many geometrical properties of the problem can be described as powers (critical exponents) of $u_c-u$, such as $\beta$, which controls the strength of the spanning cluster, and $\gamma$, which characterizes the behavior of the mean finite cluster size $S$. We show that at $u_c$ the ratio between the mean mass of the largest cluster $M_1$ and the mass of the second largest cluster $M_2$ is independent of $L$ and can be used to find $u_c$. We calculate $\beta$ from the $L$-dependence of $M_2$ and $\gamma$ from the finite size scaling of $S$. The resulting exponent $\beta$ remains close to 1 in all dimensions. The exponent $\gamma$ decreases from $\approx 3.9$ in $d=3$ to $\approx1.9$ in $d=4$ and $\approx 1.3$ in $d=5$ towards $\gamma=1$ expected in $d=6$, which is close to $\gamma=4/(d-2)$., Comment: LaTeX, 8 figures, 9 pages

Autonomous navigation is crucial for various robotics applications in agriculture. However, many existing methods depend on RTK-GPS devices, which can be susceptible to loss of radio signal or intermittent reception of corrections from the internet. Consequently, research has increasingly focused on using RGB cameras for crop-row detection, though challenges persist when dealing with grown plants. This paper introduces a LiDAR-based navigation system that can achieve crop-agnostic over-canopy autonomous navigation in row-crop fields, even when the canopy fully blocks the inter-row spacing. Our algorithm can detect crop rows across diverse scenarios, encompassing various crop types, growth stages, the presence of weeds, curved rows, and discontinuities. Without utilizing a global localization method (i.e., based on GPS), our navigation system can perform autonomous navigation in these challenging scenarios, detect the end of the crop rows, and navigate to the next crop row autonomously, providing a crop-agnostic approach to navigate an entire field. The proposed navigation system has undergone tests in various simulated and real agricultural fields, achieving an average cross-track error of 3.55cm without human intervention. The system has been deployed on a customized UGV robot, which can be reconfigured depending on the field conditions., Comment: 6 pages, 9 figures, submitted to ICRA 2025 (Under Review)

Shared gaze visualizations have been found to enhance collaboration and communication outcomes in diverse HCI scenarios including computer supported collaborative work and learning contexts. Given the importance of gaze in surgery operations, especially when a surgeon trainer and trainee need to coordinate their actions, research on the use of gaze to facilitate intra-operative coordination and instruction has been limited and shows mixed implications. We performed a field observation of 8 surgeries and an interview study with 14 surgeons to understand their visual needs during operations, informing ways to leverage and augment gaze to enhance intra-operative coordination and instruction. We found that trainees have varying needs in receiving visual guidance which are often unfulfilled by the trainers' instructions. It is critical for surgeons to control the timing of the gaze-based visualizations and effectively interpret gaze data. We suggest overlay technologies, e.g., gaze-based summaries and depth sensing, to augment raw gaze in support of surgical coordination and instruction.

This paper presents a modular, reconfigurable robot platform for robot manipulation in agriculture. While robot manipulation promises great advancements in automating challenging, complex tasks that are currently best left to humans, it is also an expensive capital investment for researchers and users because it demands significantly varying robot configurations depending on the task. Modular robots provide a way to obtain multiple configurations and reduce costs by enabling incremental acquisition of only the necessary modules. The robot we present, Hefty, is designed to be modular and reconfigurable. It is designed for both researchers and end-users as a means to improve technology transfer from research to real-world application. This paper provides a detailed design and integration process, outlining the critical design decisions that enable modularity in the mobility of the robot as well as its sensor payload, power systems, computing, and fixture mounting. We demonstrate the utility of the robot by presenting five configurations used in multiple real-world agricultural robotics applications., Comment: 8 pages, 11 figures

Videos are prominent learning materials to prepare surgical trainees before they enter the operating room (OR). In this work, we explore techniques to enrich the video-based surgery learning experience. We propose Surgment, a system that helps expert surgeons create exercises with feedback based on surgery recordings. Surgment is powered by a few-shot-learning-based pipeline (SegGPT+SAM) to segment surgery scenes, achieving an accuracy of 92\%. The segmentation pipeline enables functionalities to create visual questions and feedback desired by surgeons from a formative study. Surgment enables surgeons to 1) retrieve frames of interest through sketches, and 2) design exercises that target specific anatomical components and offer visual feedback. In an evaluation study with 11 surgeons, participants applauded the search-by-sketch approach for identifying frames of interest and found the resulting image-based questions and feedback to be of high educational value.

The detectability of the gravitational-wave signal from $r$-modes depends on the interplay between the amplification of the mode by the CFS instability and its damping due to dissipative mechanisms present in the stellar matter. The instability window of $r$-modes describes the region of stellar parameters (angular velocity, $\Omega$, and redshifted stellar temperature, $T^\infty$), for which the mode is unstable. In this study, we reexamine this problem in nonbarotropic neutron stars, taking into account the previously overlooked nonanalytic behavior (in $\Omega$) of relativistic $r$-modes and enhanced energy dissipation resulting from diffusion in superconducting stellar matter. We demonstrate that at slow rotation rates, relativistic $r$-modes exhibit weaker amplification by the CFS instability compared to Newtonian ones. However, their dissipation through viscosity and diffusion is significantly more efficient. In rapidly rotating neutron stars within the framework of general relativity, the amplification of $r$-modes by the CFS mechanism and their damping due to shear viscosity become comparable to those predicted by Newtonian theory. In contrast, the relativistic damping of the mode by diffusion and bulk viscosity remains significantly stronger than in the nonrelativistic case. Consequently, account for diffusion and general relativity leads to a substantial modification of the $r$-mode instability window compared to the Newtonian prediction. This finding is important for the interpretation of observations of rotating neutron stars, as well as for overall understanding of $r$-mode physics., Comment: 21 pages, 8 figures, accepted for publication in Physical Review D

Richmond and Richmond (American Mathematical Monthly 104 (1997), 713--719) proved the following theorem: If, in a metric space with at least five points, all triangles are degenerate, then the space is isometric to a subset of the real line. We prove that the hypothesis is unnecessarily strong: In a metric space on $n$ points, fewer than $7n^2/6$ suitably placed degenerate triangles suffice. However, fewer than $n(n-1)/2$ degenerate triangles, no matter how cleverly placed, never suffice., Comment: Supersedes arXiv:2209.14361 [math.MG]

Agricultural robotics is an active research area due to global population growth and expectations of food and labor shortages. Robots can potentially help with tasks such as pruning, harvesting, phenotyping, and plant modeling. However, agricultural automation is hampered by the difficulty in creating high resolution 3D semantic maps in the field that would allow for safe manipulation and navigation. In this paper, we build toward solutions for this issue and showcase how the use of semantics and environmental priors can help in constructing accurate 3D maps for the target application of sorghum. Specifically, we 1) use sorghum seeds as semantic landmarks to build a visual Simultaneous Localization and Mapping (SLAM) system that enables us to map 78\\% of a sorghum range on average, compared to 38% with ORB-SLAM2; and 2) use seeds as semantic features to improve 3D reconstruction of a full sorghum panicle from images taken by a robotic in-hand camera., Comment: 7 pages, 8 figures

Epsilon-near-zero (ENZ) materials, i.e., materials with a vanishing real part of the permittivity, have become an increasingly desirable platform for exploring linear and nonlinear optical phenomena in nanophotonic and on-chip environments. ENZ materials inherently enhance electric fields for properly chosen interaction scenarios, host extreme nonlinear optical effects, and lead to other intriguing phenomena. To date, studies in the optical domain have mainly focused on nanoscopically thin films of ENZ materials and their interaction with light and other nanostructured materials. Here, we experimentally and numerically explore the optical response of individual nanostructures milled into an ENZ material. For the study, we employ 3D structured light beams, allowing us to fully control polarization-dependent field enhancements enabled by a tailored illumination and a vanishing permittivity. Our studies provide insight between complex near-fields and the ENZ regime while showcasing the polarization-dependent controllability they feature. Such effects can form the basis for experimental realizations of extremely localized polarization-controlled refractive index changes, which can ultimately enable ultrafast switching processes at the level of individual nanostructures., Comment: 7 pages, 8 figures - Updated: 8 pages, 11 figures, revised abstract, re-structured figure placement and formatting, corrected figure coordinate axes, corrected experimental hole diameter labels, added additional simulation figure, revised arguments in sections 2 and 4, added supplemental document (3 pages, 4 figures), corrected typos, added co-authors, additional acknowledgements entered

Various NLP tasks require a complex hierarchical structure over nodes, where each node is a cluster of items. Examples include generating entailment graphs, hierarchical cross-document coreference resolution, annotating event and subevent relations, etc. To enable efficient annotation of such hierarchical structures, we release CHAMP, an open source tool allowing to incrementally construct both clusters and hierarchy simultaneously over any type of texts. This incremental approach significantly reduces annotation time compared to the common pairwise annotation approach and also guarantees maintaining transitivity at the cluster and hierarchy levels. Furthermore, CHAMP includes a consolidation mode, where an adjudicator can easily compare multiple cluster hierarchy annotations and resolve disagreements., Comment: EMNLP 2023

There is growing interest in automating agricultural tasks that require intricate and precise interaction with specialty crops, such as trees and vines. However, developing robotic solutions for crop manipulation remains a difficult challenge due to complexities involved in modeling their deformable behavior. In this study, we present a framework for learning the deformation behavior of tree-like crops under contact interaction. Our proposed method involves encoding the state of a spring-damper modeled tree crop as a graph. This representation allows us to employ graph networks to learn both a forward model for predicting resulting deformations, and a contact policy for inferring actions to manipulate tree crops. We conduct a comprehensive set of experiments in a simulated environment and demonstrate generalizability of our method on previously unseen trees. Videos can be found on the project website: https://kantor-lab.github.io/tree_gnn, Comment: 7 pages, 10 figures

We present a contact-based phenotyping robot platform that can autonomously insert nitrate sensors into cornstalks to proactively monitor macronutrient levels in crops. This task is challenging because inserting such sensors requires sub-centimeter precision in an environment which contains high levels of clutter, lighting variation, and occlusion. To address these challenges, we develop a robust perception-action pipeline to detect and grasp stalks, and create a custom robot gripper which mechanically aligns the sensor before inserting it into the stalk. Through experimental validation on 48 unique stalks in a cornfield in Iowa, we demonstrate our platform's capability of detecting a stalk with 94% success, grasping a stalk with 90% success, and inserting a sensor with 60% success. In addition to developing an autonomous phenotyping research platform, we share key challenges and insights obtained from deployment in the field. Our research platform is open-sourced, with additional information available at https://kantor-lab.github.io/cornbot.

Given a metric space $(X, \rho)$, we say $y$ is between $x$ and $z$ if $\rho(x,z) = \rho(x,y) + \rho(y,z)$. A metric space gives rise to a 3-uniform hypergraph that has as hyperedges those triples $\{ x,y,z \}$ where $y$ is between $x$ and $z$. Such hypergraphs are called metric and understanding them is key to the study of metric spaces. In this paper, we prove that hypergraphs where small subsets of vertices induce few edges are metric. Additionally, we adapt the notion of sparsity with respect to monotone increasing functions, classify hypergraphs that exhibit this version of sparsity and prove that they are metric., Comment: 6 pages, 16 figures

In this paper, we present a next-best-view planning approach to autonomously size apple fruitlets. State-of-the-art viewpoint planners in agriculture are designed to size large and more sparsely populated fruit. They rely on lower resolution maps and sizing methods that do not generalize to smaller fruit sizes. To overcome these limitations, our method combines viewpoint sampling around semantically labeled regions of interest, along with an attention-guided information gain mechanism to more strategically select viewpoints that target the small fruits' volume. Additionally, we integrate a dual-map representation of the environment that is able to both speed up expensive ray casting operations and maintain the high occupancy resolution required to informatively plan around the fruit. When sizing, a robust estimation and graph clustering approach is introduced to associate fruit detections across images. Through simulated experiments, we demonstrate that our viewpoint planner improves sizing accuracy compared to state of the art and ablations. We also provide quantitative results on data collected by a real robotic system in the field.

Robotic pruning of dormant grapevines is an area of active research in order to promote vine balance and grape quality, but so far robotic efforts have largely focused on planar, simplified vines not representative of commercial vineyards. This paper aims to advance the robotic perception capabilities necessary for pruning in denser and more complex vine structures by extending plant skeletonization techniques. The proposed pipeline generates skeletal grapevine models that have lower reprojection error and higher connectivity than baseline algorithms. We also show how 3D and skeletal information enables prediction accuracy of pruning weight for dense vines surpassing prior work, where pruning weight is an important vine metric influencing pruning site selection., Comment: 6 pages, IROS 2023 Computer Vision for Automation

Reliable real-world deployment of reinforcement learning (RL) methods requires a nuanced understanding of their strengths and weaknesses and how they compare to those of humans. Human-machine systems are becoming more prevalent and the design of these systems relies on a task-oriented understanding of both human learning (HL) and RL. Thus, an important line of research is characterizing how the structure of a learning task affects learning performance. While increasingly complex benchmark environments have led to improved RL capabilities, such environments are difficult to use for the dedicated study of task structure. To address this challenge we present a learning environment built to support rigorous study of the impact of task structure on HL and RL. We demonstrate the environment's utility for such study through example experiments in task structure that show performance differences between humans and RL algorithms., Comment: 9 pages, 4 figures, additional content in appendix

Key Point Analysis (KPA) has been recently proposed for deriving fine-grained insights from collections of textual comments. KPA extracts the main points in the data as a list of concise sentences or phrases, termed key points, and quantifies their prevalence. While key points are more expressive than word clouds and key phrases, making sense of a long, flat list of key points, which often express related ideas in varying levels of granularity, may still be challenging. To address this limitation of KPA, we introduce the task of organizing a given set of key points into a hierarchy, according to their specificity. Such hierarchies may be viewed as a novel type of Textual Entailment Graph. We develop ThinkP, a high quality benchmark dataset of key point hierarchies for business and product reviews, obtained by consolidating multiple annotations. We compare different methods for predicting pairwise relations between key points, and for inferring a hierarchy from these pairwise predictions. In particular, for the task of computing pairwise key point relations, we achieve significant gains over existing strong baselines by applying directional distributional similarity methods to a novel distributional representation of key points, and further boost performance via weak supervision., Comment: ACL 2023

We present MeeQA, a dataset for natural-language question answering over meeting transcripts. It includes real questions asked during meetings by its participants. The dataset contains 48K question-answer pairs, extracted from 422 meeting transcripts, spanning multiple domains. Questions in transcripts pose a special challenge as they are not always clear, and considerable context may be required in order to provide an answer. Further, many questions asked during meetings are left unanswered. To improve baseline model performance on this type of questions, we also propose a novel loss function, \emph{Flat Hierarchical Loss}, designed to enhance performance over questions with no answer in the text. Our experiments demonstrate the advantage of using our approach over standard QA models.

Modern Natural Language Generation (NLG) models come with massive computational and storage requirements. In this work, we study the potential of compressing them, which is crucial for real-world applications serving millions of users. We focus on Knowledge Distillation (KD) techniques, in which a small student model learns to imitate a large teacher model, allowing to transfer knowledge from the teacher to the student. In contrast to much of the previous work, our goal is to optimize the model for a specific NLG task and a specific dataset. Typically in real-world applications, in addition to labeled data there is abundant unlabeled task-specific data, which is crucial for attaining high compression rates via KD. In this work, we conduct a systematic study of task-specific KD techniques for various NLG tasks under realistic assumptions. We discuss the special characteristics of NLG distillation and particularly the exposure bias problem. Following, we derive a family of Pseudo-Target (PT) augmentation methods, substantially extending prior work on sequence-level KD. We propose the Joint-Teaching method, which applies word-level KD to multiple PTs generated by both the teacher and the student. Finally, we validate our findings in an extreme setup with no labeled examples using GPT-4 as the teacher. Our study provides practical model design observations and demonstrates the effectiveness of PT training for task-specific KD in NLG.