Search

Humanoid robots can, in principle, use their legs to go almost anywhere. Developing controllers capable of traversing diverse terrains, however, remains a considerable challenge. Classical controllers are hard to generalize broadly while the learning-based methods have primarily focused on gentle terrains. Here, we present a learning-based approach for blind humanoid locomotion capable of traversing challenging natural and man-made terrain. Our method uses a transformer model to predict the next action based on the history of proprioceptive observations and actions. The model is first pre-trained on a dataset of flat-ground trajectories with sequence modeling, and then fine-tuned on uneven terrain using reinforcement learning. We evaluate our model on a real humanoid robot across a variety of terrains, including rough, deformable, and sloped surfaces. The model demonstrates robust performance, in-context adaptation, and emergent terrain representations. In real-world case studies, our humanoid robot successfully traversed over 4 miles of hiking trails in Berkeley and climbed some of the steepest streets in San Francisco., Comment: Project page: https://humanoid-challenging-terrain.github.io

Single Event Latch-up (SEL) is one of the prime concerns for CMOS ICs used in space systems. Galactic Cosmic Rays or Solar Energetic Particles (SEP) may trigger the parasitic latch up circuit in CMOS ICs and cause increase in current beyond the safe limits thereby presenting a threat of permanent failure of the IC. Mitigation of the SEL is always a challenging task. The conventional mitigation approaches inherently introduce some response time which presents an uncertainty because during this response time the current may exceed the safe current limits. This paper presents a novel circuit based on MOSFETs which provides end-to-end complete solution of detecting SEL, limiting the current below the set threshold and executing power cycling to restore the normal functioning of the CMOS IC. The proposed circuit has been simulated in MULTISIM and the simulation results match very well with the expected behavior of (i)current limiting and (ii) the total time duration taken in power cycling to bring the SEL sensitive device back to its normal operational state. This circuit can be harnessed by spacecraft system designers to overcome the catastrophic threat of SEL posed by space radiation environment.

Current open-vocabulary scene graph generation algorithms highly rely on both 3D scene point cloud data and posed RGB-D images and thus have limited applications in scenarios where RGB-D images or camera poses are not readily available. To solve this problem, we propose Point2Graph, a novel end-to-end point cloud-based 3D open-vocabulary scene graph generation framework in which the requirement of posed RGB-D image series is eliminated. This hierarchical framework contains room and object detection/segmentation and open-vocabulary classification. For the room layer, we leverage the advantage of merging the geometry-based border detection algorithm with the learning-based region detection to segment rooms and create a "Snap-Lookup" framework for open-vocabulary room classification. In addition, we create an end-to-end pipeline for the object layer to detect and classify 3D objects based solely on 3D point cloud data. Our evaluation results show that our framework can outperform the current state-of-the-art (SOTA) open-vocabulary object and room segmentation and classification algorithm on widely used real-scene datasets., Comment: 8 pages, 9 figures

The seminal Erd\H{o}s--Ko--Rado (EKR) theorem states that if $\mathcal{F}$ is a family of $k$-subsets of an $n$-element set $X$ for $k\leq n/2$ such that every pair of subsets in $\mathcal{F}$ has a nonempty intersection, then $\mathcal{F}$ can be no bigger than the trivially intersecting family obtained by including all $k$-subsets of $X$ that contain a fixed element $x\in X$. This family is called the star centered at $x$. In this paper, we formulate and prove an EKR theorem for intersecting families of subgraphs of the perfect matching graph, the graph consisting of $n$ disjoint edges. This can be considered a generalization not only of the aforementioned EKR theorem but also of a signed variant of it, first stated by Meyer (1974), and proved separately by Deza--Frankl (1983) and Bollob\'as--Leader (1997). The proof of our main theorem relies on a novel extension of Katona's beautiful cycle method., Comment: 10 pages, 2 figures

In many applications, researchers seek to identify overlapping entities across multiple data files. Record linkage algorithms facilitate this task, in the absence of unique identifiers. As these algorithms rely on semi-identifying information, they may miss records that represent the same entity, or incorrectly link records that do not represent the same entity. Analysis of linked files commonly ignores such linkage errors, resulting in biased, or overly precise estimates of the associations of interest. We view record linkage as a missing data problem, and delineate the linkage mechanisms that underpin analysis methods with linked files. Following the missing data literature, we group these methods under three categories: likelihood and Bayesian methods, imputation methods, and weighting methods. We summarize the assumptions and limitations of the methods, and evaluate their performance in a wide range of simulation scenarios., Comment: Accepted manuscript, to be published in Statistical Science

The electronic properties of moir\'e heterostructures depend sensitively on the relative orientation between layers of the stack. For example, near-magic-angle twisted bilayer graphene (TBG) commonly shows superconductivity, yet a TBG sample with one of the graphene layers rotationally aligned to a hexagonal Boron Nitride (hBN) cladding layer provided the first experimental observation of orbital ferromagnetism. To create samples with aligned graphene/hBN, researchers often align edges of exfoliated flakes that appear straight in optical micrographs. However, graphene or hBN can cleave along either zig-zag or armchair lattice directions, introducing a 30 degree ambiguity in the relative orientation of two flakes. By characterizing the crystal lattice orientation of exfoliated flakes prior to stacking using Raman and second-harmonic generation for graphene and hBN, respectively, we unambiguously align monolayer graphene to hBN at a near-0 degree, not 30 degree, relative twist angle. We confirm this alignment by torsional force microscopy (TFM) of the graphene/hBN moir\'e on an open-face stack, and then by cryogenic transport measurements, after full encapsulation with a second, non-aligned hBN layer. This work demonstrates a key step toward systematically exploring the effects of the relative twist angle between dissimilar materials within moir\'e heterostructures., Comment: 40 pages, 15 figures

As the number of Persons with Disabilities (PWD), particularly those with one or more physical impairments, increases, there is an increasing demand for assistive robotic technologies that can support independent mobility in the built environment and reduce the burden on caregivers. Current assistive mobility platforms (e.g., robotic wheelchairs) often fail to incorporate user preferences and control, leading to reduced trust and efficiency. Existing shared control algorithms do not allow the incorporation of the user control preferences inside the navigation framework or the path planning algorithm. In addition, existing dynamic local planner algorithms for robotic wheelchairs do not take into account the social spaces of people, potentially leading such platforms to infringe upon these areas and cause discomfort. To address these concerns, this work introduces a novel socially-aware shared autonomy-based navigation system for assistive mobile robotic platforms. Our navigation framework comprises a Global Planner and a Local Planner. To implement the Global Planner, the proposed approach introduces a novel User Preference Field (UPF) theory within its global planning framework, explicitly acknowledging user preferences to adeptly navigate away from congested areas. For the Local Planner, we propose a Socially-aware Shared Control-based Model Predictive Control with Dynamic Control Barrier Function (SS-MPC-DCBF) to adjust movements in real-time, integrating user preferences for safer, more autonomous navigation. Evaluation results show that our Global Planner aligns closely with user preferences compared to baselines, and our Local Planner demonstrates enhanced safety and efficiency in dynamic and static scenarios. This integrated approach fosters trust and autonomy, crucial for the acceptance of assistive mobility technologies in the built environment., Comment: 42 pages, 14 figures

Vulnerable Road Users (VRUs) present a significant challenge for road safety due to the frequent unpredictability of their behaviors. In typical Intelligent Transportation Systems, vision-based approaches supported by networked cameras are often used to anticipate VRUs motion intentions and trajectories. However, several limitations posed by occlusions and distractions set a boundary for the efficacy of such methods. To address these challenges, this study introduces a framework that leverages data collected using wearable neurophysiological sensors on VRUs to integrate them seamlessly into the Vehicle-to-Everything communication framework. This integration empowers VRUs to autonomously broadcast their intended movements to other road agents, especially autonomous vehicles, thereby bridging a critical gap in current vehicular communication systems. To validate this concept, we conducted an experiment involving 12 participants, from whom EEG signals were collected as they engaged in road-crossing decisions within simulated environments. Employing Hidden Markov Models, we identified four cognitive stages intrinsic to a pedestrian's decision-making process. Our statistical analysis further revealed significant variations in EEG activities across these stages, shedding light on the neural correlates and cognitive dynamics underpinning pedestrian road-crossing behavior. We then developed a predictive cognitive model using dynamic time warping and K-nearest neighbors algorithms, optimized through a data-driven sliding window approach. This model demonstrated high predictive accuracy, evidenced by an Area Under the Curve of 0.91, indicating its capability to anticipate pedestrian road-crossing actions approximately 1 second in advance of any pedestrian movement. This research paves the way for a novel VRU-Vehicle interaction paradigm and signifies a shift towards a forward-thinking ecosystem., Comment: 33 pages, 15 figures

In the construction industry, where work environments are complex, unstructured and often dangerous, the implementation of Human-Robot Collaboration (HRC) is emerging as a promising advancement. This underlines the critical need for intuitive communication interfaces that enable construction workers to collaborate seamlessly with robotic assistants. This study introduces a conversational Virtual Reality (VR) interface integrating multimodal interaction to enhance intuitive communication between construction workers and robots. By integrating voice and controller inputs with the Robot Operating System (ROS), Building Information Modeling (BIM), and a game engine featuring a chat interface powered by a Large Language Model (LLM), the proposed system enables intuitive and precise interaction within a VR setting. Evaluated by twelve construction workers through a drywall installation case study, the proposed system demonstrated its low workload and high usability with succinct command inputs. The proposed multimodal interaction system suggests that such technological integration can substantially advance the integration of robotic assistants in the construction industry., Comment: 39 pages, 16 figures, 5 tables

We cast real-world humanoid control as a next token prediction problem, akin to predicting the next word in language. Our model is a causal transformer trained via autoregressive prediction of sensorimotor trajectories. To account for the multi-modal nature of the data, we perform prediction in a modality-aligned way, and for each input token predict the next token from the same modality. This general formulation enables us to leverage data with missing modalities, like video trajectories without actions. We train our model on a collection of simulated trajectories coming from prior neural network policies, model-based controllers, motion capture data, and YouTube videos of humans. We show that our model enables a full-sized humanoid to walk in San Francisco zero-shot. Our model can transfer to the real world even when trained on only 27 hours of walking data, and can generalize to commands not seen during training like walking backward. These findings suggest a promising path toward learning challenging real-world control tasks by generative modeling of sensorimotor trajectories.

Assigning repetitive and physically-demanding construction tasks to robots can alleviate human workers's exposure to occupational injuries. Transferring necessary dexterous and adaptive artisanal construction craft skills from workers to robots is crucial for the successful delegation of construction tasks and achieving high-quality robot-constructed work. Predefined motion planning scripts tend to generate rigid and collision-prone robotic behaviors in unstructured construction site environments. In contrast, Imitation Learning (IL) offers a more robust and flexible skill transfer scheme. However, the majority of IL algorithms rely on human workers to repeatedly demonstrate task performance at full scale, which can be counterproductive and infeasible in the case of construction work. To address this concern, this paper proposes an immersive, cloud robotics-based virtual demonstration framework that serves two primary purposes. First, it digitalizes the demonstration process, eliminating the need for repetitive physical manipulation of heavy construction objects. Second, it employs a federated collection of reusable demonstrations that are transferable for similar tasks in the future and can thus reduce the requirement for repetitive illustration of tasks by human agents. Additionally, to enhance the trustworthiness, explainability, and ethical soundness of the robot training, this framework utilizes a Hierarchical Imitation Learning (HIL) model to decompose human manipulation skills into sequential and reactive sub-skills. These two layers of skills are represented by deep generative models, enabling adaptive control of robot actions. By delegating the physical strains of construction work to human-trained robots, this framework promotes the inclusion of workers with diverse physical capabilities and educational backgrounds within the construction industry.

In a stack of atomically-thin Van der Waals layers, introducing interlayer twist creates a moir\'e superlattice whose period is a function of twist angle. Changes in that twist angle of even hundredths of a degree can dramatically transform the system's electronic properties. Setting a precise and uniform twist angle for a stack remains difficult, hence determining that twist angle and mapping its spatial variation is very important. Techniques have emerged to do this by imaging the moir\'e, but most of these require sophisticated infrastructure, time-consuming sample preparation beyond stack synthesis, or both. In this work, we show that Torsional Force Microscopy (TFM), a scanning probe technique sensitive to dynamic friction, can reveal surface and shallow subsurface structure of Van der Waals stacks on multiple length scales: the moir\'es formed between bi-layers of graphene and between graphene and hexagonal boron nitride (hBN), and also the atomic crystal lattices of graphene and hBN. In TFM, torsional motion of an AFM cantilever is monitored as it is actively driven at a torsional resonance while a feedback loop maintains contact at a set force with the sample surface. TFM works at room temperature in air, with no need for an electrical bias between the tip and the sample, making it applicable to a wide array of samples. It should enable determination of precise structural information including twist angles and strain in moir\'e superlattices and crystallographic orientation of VdW flakes to support predictable moir\'e heterostructure fabrication., Comment: 31 pages, 14 figures and 1 table including supplementary materials

In many healthcare and social science applications, information about units is dispersed across multiple data files. Linking records across files is necessary to estimate the associations of interest. Common record linkage algorithms only rely on similarities between linking variables that appear in all the files. Moreover, analysis of linked files often ignores errors that may arise from incorrect or missed links. Bayesian record linking methods allow for natural propagation of linkage error, by jointly sampling the linkage structure and the model parameters. We extend an existing Bayesian record linkage method to integrate associations between variables exclusive to each file being linked. We show analytically, and using simulations, that this method can improve the linking process, and can yield accurate inferences. We apply the method to link Meals on Wheels recipients to Medicare Enrollment records.

Provided is a comprehensive description of a band gap thermodynamic model, which predicts and explains key features of photosegregation in lead-based, mixed-halide perovskites. The model provides a prescription for illustrating halide migration driven by photocarrier energies. Where possible, model predictions are compared to experimental results. Free energy derivations are provided for three assumptions: (1) halide mixing in the dark, (2) a fixed number of photogenerated carriers funneling to and localizing in low band gap inclusions of the alloy, and (3) the statistical occupancy of said inclusions from a bath of thermalized photocarriers in the parent material. Model predictions include: excitation intensity ($I_{\textrm{exc}}$)-dependent terminal halide stoichiometries ($x_{\textrm{terminal}}$), excitation intensity thresholds ($I_{\textrm{exc,threshold}}$) below which photosegregation is suppressed, reduced segregation in nanocrystals as compared to thin films, the possibility to kinetically manipulate photosegregation rates via control of underlying mediators, asymmetries in forward and reverse photosegregation rate constants/activation energies, and a preference for high band gap products to recombine with the parent phase. What emerges is a cohesive framework for understanding ubiquitous photosegregation in mixed-halide perovskites and a rational basis by which to manage the phenomenon.

We develop a method to learn about treatment effects in multiple treatment models with discrete-valued instruments. We allow selection into treatment to be governed by a general class of threshold crossing models that permits multidimensional unobserved heterogeneity. Under a semi-parametric restriction on the distribution of unobserved heterogeneity, we show how a sequence of linear programs can be used to compute sharp bounds for a number of treatment effect parameters when the marginal treatment response functions underlying them remain nonparametric or are additionally parameterized.

The introduction of robots is widely considered to have significant potential of alleviating the issues of worker shortage and stagnant productivity that afflict the construction industry. However, it is challenging to use fully automated robots in complex and unstructured construction sites. Human-Robot Collaboration (HRC) has shown promise of combining human workers' flexibility and robot assistants' physical abilities to jointly address the uncertainties inherent in construction work. When introducing HRC in construction, it is critical to recognize the importance of teamwork and supervision in field construction and establish a natural and intuitive communication system for the human workers and robotic assistants. Natural language-based interaction can enable intuitive and familiar communication with robots for human workers who are non-experts in robot programming. However, limited research has been conducted on this topic in construction. This paper proposes a framework to allow human workers to interact with construction robots based on natural language instructions. The proposed method consists of three stages: Natural Language Understanding (NLU), Information Mapping (IM), and Robot Control (RC). Natural language instructions are input to a language model to predict a tag for each word in the NLU module. The IM module uses the result of the NLU module and building component information to generate the final instructional output essential for a robot to acknowledge and perform the construction task. A case study for drywall installation is conducted to evaluate the proposed approach. The obtained results highlight the potential of using natural language-based interaction to replicate the communication that occurs between human workers within the context of human-robot teams.

Williamson's theorem is well known for symmetric matrices. In this paper, we state and re-derive some of the cases of Williamson's theorem for symmetric positive-semi definite matrices and symmetric matrices having negative index 1, due to H\"ormander. We prove theorems that guarantee conditions under which two symmetric positive-definite matrices can be simultaneously diagonalized in the sense of Williamson's theorem and their corollaries. Finally, we provide an application of this result to physical systems and another connecting the decompositions for the degenerate and non-degenerate cases, involving phase space constraints that we later apply to phase space cylinders and ellipsoids via symplectic capacities., Comment: 29 pages

The introduction of assistive construction robots can significantly alleviate physical demands on construction workers while enhancing both the productivity and safety of construction projects. Leveraging a Building Information Model (BIM) offers a natural and promising approach to driving robotic construction workflows. However, because of uncertainties inherent in construction sites, such as discrepancies between the as-designed and as-built components, robots cannot solely rely on a BIM to plan and perform field construction work. Human workers are adept at improvising alternative plans with their creativity and experience and thus can assist robots in overcoming uncertainties and performing construction work successfully. In such scenarios, it is critical to continuously update the BIM as work processes unfold so that it includes as-built information for the ensuing construction and maintenance tasks. This research introduces an interactive closed-loop digital twin framework that integrates a BIM into human-robot collaborative construction workflows. The robot's functions are primarily driven by the BIM, but it adaptively adjusts its plans based on actual site conditions, while the human co-worker oversees and supervises the process. When necessary, the human co-worker intervenes to help the robot overcome the encountered uncertainties. A drywall installation case study is conducted to verify the proposed workflow. In addition, experiments are carried out to evaluate the system performance using an industrial robotic arm in a research laboratory setting that mimics a construction site and in the Gazebo simulation. Integrating the flexibility of human workers and the autonomy and accuracy afforded by the BIM, the proposed framework offers significant promise of increasing the robustness of construction robots in the performance of field construction work.