Search

Autonomous vehicles have been envisioned for more than 100 years. One of the first suggestions was a front cover of Scientific America back in 1916. Today, it is possible to get cars that drive autonomously for extended distances. We are also starting to see micro-mobility solutions, such as the Nuro vehicles for pizza delivery. Building autonomous cars that can operate in urban environments with a diverse set of road-users is far from trivial. Early 2018 the Contextual Robotics Institute at UC San Diego launched an effort to build a full stack autonomous vehicle for micro-mobility. The motivations were diverse: i) development of a system for operation in an environment with many pedestrians, ii) design of a system that does not rely on dense maps (or HD-maps as they are sometimes named), iii) design strategies to build truly robust systems, and iv) a framework to educate next-generation engineers. In this paper, we present the research effort of design, prototyping, and evaluation of such a vehicle. From the evaluation, several research directions are explored to account for shortcomings. Lessons and issues for future work are additionally drawn from this work.

The manufacturing world is moving towards a setup with high mix and medium volume. This requires new production paradigms. The World Robotics Challenge (2018 & 2020) was designed to challenge teams to design systems that are easy to adapt to new tasks and to ensure robust operation in a semi-structured environment. We present a layered strategy to transform missions into tasks and actions and provide a set of strategies to address simple and complex failures. We propose a model for characterizing failures using this model and discuss repairs. Simple failures are by far the most common in our WRC system and we also present how we repaired them.

Background Assigning meaningful probabilities of SARS-CoV-2 infection risk presents a diagnostic challenge across the continuum of care. Objective The aim of this study was to develop and clinically validate an adaptable, personalized diagnostic model to assist clinicians in ruling in and ruling out COVID-19 in potential patients. We compared the diagnostic performance of probabilistic, graphical, and machine learning models against a previously published benchmark model. Methods We integrated patient symptoms and test data using machine learning and Bayesian inference to quantify individual patient risk of SARS-CoV-2 infection. We trained models with 100,000 simulated patient profiles based on 13 symptoms and estimated local prevalence, imaging, and molecular diagnostic performance from published reports. We tested these models with consecutive patients who presented with a COVID-19–compatible illness at the University of California San Diego Medical Center over the course of 14 days starting in March 2020. Results We included 55 consecutive patients with fever (n=43, 78%) or cough (n=42, 77%) presenting for ambulatory (n=11, 20%) or hospital care (n=44, 80%). In total, 51% (n=28) were female and 49% (n=27) were aged Conclusions Decision support models that incorporate symptoms and available test results can help providers diagnose SARS-CoV-2 infection in real-world settings.

Calibration of sensors is fundamental to robust performance for intelligent vehicles. In natural environments, disturbances can easily challenge calibration. One possibility is to use natural objects of known shape to recalibrate sensors. An approach based on recognition of traffic signs, such as stop signs, and use of them for recalibration of cameras is presented. The approach is based on detection, geometry estimation, calibration, and recursive updating. Results from natural environments are presented that clearly show convergence and improved performance., 7 pages, 7 figures, 1 table, Accepted to ICRA 2021

BackgroundAssigning meaningful probabilities of SARS-CoV-2 infection risk presents a diagnostic challenge across the continuum of care.MethodsWe integrated patient symptom and test data using machine learning and Bayesian inference to quantify individual patient risk of SARS-CoV-2 infection. We trained models with 100,000 simulated patient profiles based on thirteen symptoms, estimated local prevalence, imaging, and molecular diagnostic performance from published reports. We tested these models with consecutive patients who presented with a COVID-19 compatible illness at the University of California San Diego Medical Center over 14 days starting in March 2020.ResultsWe included 55 consecutive patients with fever (78%) or cough (77%) presenting for ambulatory (n=11) or hospital care (n=44). 51% (n=28) were female, 49% were age ConclusionsDecision support models that incorporate symptoms and available test results can help providers diagnose SARS-CoV-2 infection in real world settings.

Recent advancements in statistical learning and computational abilities have enabled autonomous vehicle technology to develop at a much faster rate. While many of the architectures previously introduced are capable of operating under highly dynamic environments, many of these are constrained to smaller-scale deployments, require constant maintenance due to the associated scalability cost with high-definition (HD) maps, and involve tedious manual labeling. As an attempt to tackle this problem, we propose to fuse image and pre-built point cloud map information to perform automatic and accurate labeling of static landmarks such as roads, sidewalks, crosswalks, and lanes. The method performs semantic segmentation on 2D images, associates the semantic labels with point cloud maps to accurately localize them in the world, and leverages the confusion matrix formulation to construct a probabilistic semantic map in bird's eye view from semantic point clouds. Experiments from data collected in an urban environment show that this model is able to predict most road features and can be extended for automatically incorporating road features into HD maps with potential future work directions., 6 pages, 7 figures, IROS 2020

COVID-19 may drive sustained research in robotics to address risks of infectious diseases.

In recent years, there has been a rapid increase in the number of service robots deployed for aiding people in their daily activities. Unfortunately, most of these robots require human input for training in order to do tasks in indoor environments. Successful domestic navigation often requires access to semantic information about the environment, which can be learned without human guidance. In this paper, we propose a set of DEDUCE - Diverse scEne Detection methods in Unseen Challenging Environments algorithms which incorporate deep fusion models derived from scene recognition systems and object detectors. The five methods described here have been evaluated on several popular recent image datasets, as well as real-world videos acquired through multiple mobile platforms. The final results show an improvement over the existing state-of-the-art visual place recognition systems., Paper accepted at IROS'19

Scaling up the software system on service robots increases the maintenance burden of developers and the risk of resource contention of the computer embedded on robots. As a result, developers spend much time on configuring, deploying, and monitoring the robot software system; robots may utilize significant computer resources when all software processes are running. We present Rorg, a Linux container-based scheme to manage, schedule, and monitor software components on service robots. Although Linux containers are already widely-used in cloud environments, this technique is challenging to efficiently adopt in service robot systems due to multi-tasking, resource constraints and performance requirements. To pave the way of Linux containers on service robots in an efficient manner, we present a programmable container management interface and a resource time-sharing mechanism incorporated with the Robot Operating System (ROS). Rorg allows developers to pack software into self-contained images and runs them in isolated environments using Linux containers; it also allows the robot to turn on and off software components on demand to avoid resource contention. We evaluate Rorg with a long-term autonomous tour guide robot: It manages 41 software components on the robot and relieved our maintenance burden, and it also reduces CPU load by 45.5% and memory usage by 16.5% on average.

In recent years, predicting driver's focus of attention has been a very active area of research in the autonomous driving community. Unfortunately, existing state-of-the-art techniques achieve this by relying only on human gaze information, thereby ignoring scene semantics. We propose a novel Semantics Augmented GazE (SAGE) detection approach that captures driving specific contextual information, in addition to the raw gaze. Such a combined attention mechanism serves as a powerful tool to focus on the relevant regions in an image frame in order to make driving both safe and efficient. Using this, we design a complete saliency prediction framework - SAGE-Net, which modifies the initial prediction from SAGE by taking into account vital aspects such as distance to objects (depth), ego vehicle speed, and pedestrian crossing intent. Exhaustive experiments conducted through four popular saliency algorithms show that on $\mathbf{49/56\text{ }(87.5\%)}$ cases - considering both the overall dataset and crucial driving scenarios, SAGE outperforms existing techniques without any additional computational overhead during the training process. The augmented dataset along with the relevant code are available as part of the supplementary material., Comment: Paper accepted at CVPR-2020

We identify 10 exciting robotics developments and technologies, ranging from original research that may change the future of robotics to commercial products that enable basic science and drive industrial and medical innovations.

Service robots perform flawless demos while their developers are keeping a close eye, but they often fail when working autonomously, especially in a long-term deployment. To study failure modes and human-robot interaction patterns in long-term deployments, we built TritonBot, a long-term autonomy robot working as a building receptionist and a tour guide. It recognizes people's face, talks to them, and guides people to the labs and facilities in an office building. This paper presents the design of TritonBot and the lessons we learned from the first-month deployment with respect to technical and human-robot interaction aspects. TritonBot and its variant BoxBot have worked for 108.7 hours, actively interacted with people for 22.1 hours, greeted people 2950 times, guided 150 tours, and traveled 9.9 kilometers. We share the components of TritonBot using an open licence to help the community to replicate the TritonBot platform and inspire long-term autonomy and human-robot interaction research.

SROS1 is a proposed addition to the ROS1 API and ecosystem to support modern cryptography and security measures. An overview of current progress will be presented, explaining each major advancement, including: over-the-wire cryptography for all data transport, namespaced access control enforcing graph policies/restrictions, and finally process profiles using Linux Security Modules to harden a node’s resource access. This chapter not only seeks to raise community awareness of the vulnerabilities in ROS1, but to provide clear instruction along designed patterns of development for using proposed solutions provided by SROS1 to advance the state of security for open source robotics subsystems.

We present an object pose estimation approach exploiting both geometric depth and photometric color information available from an RGB-D sensor. In contrast to various efforts relying on object segmentation with a known background structure, our approach does not depend on the segmentation and thus exhibits superior performance in unstructured environments. Inspired by a voting-based approach employing an oriented point pair feature, we present a voting-based approach which further incorporates color information from the RGB-D sensor and which exploits parallel power of the modern parallel computing architecture. The proposed approach is extensively evaluated with three state-of-the-art approaches on both synthetic and real datasets, and our approach outperforms the other approaches in terms of both computation time and accuracy. A point pair feature describing both geometric shape and photometric color and its application to a voting-based 6-DOF object pose estimation.Parallel algorithms significantly accelerating the voting process on modern GPU architectures.Effective in unstructured scenes in which the prevailing segmentation-based approaches may not be applicable.

Security of robotics systems, as well as of the related middleware infrastructures, is a critical issue for industrial and domestic IoT, and it needs to be continuously assessed throughout the whole development lifecycle. The next generation open source robotic software stack, ROS2, is now targeting support for Secure DDS, providing the community with valuable tools for secure real world robotic deployments. In this work, we introduce a framework for procedural provisioning access control policies for robotic software, as well as for verifying the compliance of generated transport artifacts and decision point implementations.

We present an efficient and scalable algorithm for segmenting 3D RGBD point clouds by combining depth, color, and temporal information using a multistage, hierarchical graph-based approach. Our algorithm processes a moving window over several point clouds to group similar regions over a graph, resulting in an initial over-segmentation. These regions are then merged to yield a dendrogram using agglomerative clustering via a minimum spanning tree algorithm. Bipartite graph matching at a given level of the hierarchical tree yields the final segmentation of the point clouds by maintaining region identities over arbitrarily long periods of time. We show that a multistage segmentation with depth then color yields better results than a linear combination of depth and color. Due to its incremental processing, our algorithm can process videos of any length and in a streaming pipeline. The algorithm's ability to produce robust, efficient segmentation is demonstrated with numerous experimental results on challenging sequences from our own as well as public RGBD data sets., Comment: CVPR 2014

Robots are becoming increasingly prevalent in our society in forms where they are assisting or interacting with humans in a variety of environments, and thus they must have the ability to sense and detect objects by touch. An ongoing challenge for soft robots has been incorporating flexible sensors that can recognize complex motions and close the loop for tactile sensing. We present sensor skins that enable haptic object visualization when integrated on a soft robotic gripper that can twist an object. First, we investigate how the design of the actuator modules impact bend angle and motion. Each soft finger is molded using a silicone elastomer, and consists of three pneumatic chambers which can be inflated independently to achieve a range of complex motions. Three fingers are combined to form a soft robotic gripper. Then, we manufacture and attach modular, flexible sensory skins on each finger to measure deformation and contact. These sensor measurements are used in conjunction with an analytical model to construct 2D and 3D tactile object models. Our results are a step towards soft robot grippers capable of a complex range of motions and proprioception, which will help future robots better understand the environments with which they interact, and has the potential to increase physical safety in human-robot interaction. Please see the accompanying video for additional details.

This paper presents a taxonomy of benchmark manipulation tasks for service robots. Our contributions are threefold: (1) A review of relevant literature regarding manipulation tests in the robotics domain and related fields, such as physical therapy, assistive technologies and prosthetics. (2) Guidelines to design useful testing protocols to evaluate manipulation performance. (3) A proposed general taxonomy of benchmark manipulation tasks and sample tests per each class.

Situational awareness in rescue operations can be provided by teams of autonomous mobile robots. Human operators are required to teleoperate the current generation of mobile robots for such applications; however, teleoperation is increasingly difficult as the number of robots is expanded. As the number of robots is increased, each robot may also interfere with one another and eventually decrease mapping performance. As presented here, through careful consideration of robot team coordination and exploration strategy, large numbers of mobile robots can be allocated to accomplish the mapping task more quickly and accurately. We present both the coordination and exploration strategies and present results from experiments in simulation as well as with up to nine mobile platforms.

We propose a multi robot SLAM approach that uses 3D objects as landmarks for localization and mapping. The approach is fully distributed in that the robots only communicate during rendezvous and there is no centralized server gathering the data. Moreover, it leverages local computation at each robot (e.g., object detection and object pose estimation) to reduce the communication burden. We show that object-based representations reduce the memory requirements and information exchange among robots, compared to point-cloud-based representations; this enables operation in severely bandwidth-constrained scenarios. We test the approach in simulations and field tests, demonstrating its advantages over related techniques: our approach is as accurate as a centralized method, scales well to large teams, and is resistant to noise.

In this tutorial chapter we’ll cover the growing intersection between ROS and Docker, showcasing new development tools and strategies to advance robotic software design and deployment within a ROS/Gazebo context by utilizing advances in Linux containers. Tutorial examples here will focus on robotics software development for education, research, and industry, specifically: constructing repeatable and reproducible environments, leveraging software defined networking, as well as running and shipping portable ROS applications.

In this paper we present the first planner for the problem of Navigation Among Movable Obstacles (NAMO) on a real robot that can handle environments with under-specified object dynamics. This result makes use of recent progress from two threads of the Reinforcement Learning literature. The first is a hierarchical Markov-Decision Process formulation of the NAMO problem designed to handle dynamics uncertainty. The second is a physics-based Reinforcement Learning framework which offers a way to ground this uncertainty in a compact model space that can be efficiently updated from data received by the robot online. Our results demonstrate the ability of a robot to adapt to unexpected object behavior in a real office scenario.

In this paper, we present an efficient assembly sequence generation method for robot programming, which can reduce the complexity of assembly sequence generation for products within a product family. Today, principle markets for robot assembly automation (e.g., automobile and consumer electronics) have a relatively short product run rate, ranging from six to twelve months. To accommodate this rapidly changing production demand, in most cases, manufacturers design a series of new products within a product family (i.e., a group of products derived from a common product platform) so that they have similar assembly sequences. Seemingly robot programming for a new product within a product family does not require extensive programming adjustments, but it is still complex and manufacturers spend large amount time and money in programming robots for each new product. To solve this problem, we present a new assembly sequence generation method. With a case-based planner, we generate feasible and similar-to-real assembly sequence candidates for a new product from assembly sequences of old products within a product family. Then, we use geometric information extracted from its CAD model to evaluate those candidate sequences. Unlike those used by previous studies, our method can generate an assembly sequence that is more suitable for the concept of a reconfigurable assembly system for general-purpose robots (assembly sequence generation for robot programming); more similar to assembly sequences used in real assembly systems (case-based planning); and more properly evaluated for robot assembly (a CAD model). We apply the proposed method to a toy airplane assembly scenario and demonstrate its feasibility.

Existing Simultaneous Localization and Mapping systems require an extensive manual pre-calibration process. Non-manual calibration procedures use manipulators to create known patterns in order to estimate the unknown calibration. Calibration is often time-consuming and involves humans performing repetitive tasks such as aligning a known calibration target at different poses with respect to the sensor. We propose an algorithm that plans a trajectory which actively reduces the uncertainty of the robot's calibration given a rough initial calibration estimate. Calibration is performed autonomously in a previously unknown environment by maintaining the belief over landmarks, poses, and the calibration parameters. We present experimental results to demonstrate the approach's ability to autonomously calibrate the exteroceptive sensor in simulated and real environments. We show that even a greedy approach can reduce the effort needed to perform calibration every time the robot is reconfigured for autonomous tasks and mitigates the possibility of human error added into the calibration.

In this paper we propose an approach to robot grasp prioritization based on a combined arm-and-hand metric. Most traditional approaches evaluate grasps based on hand-centric metrics such as force-closure, finger spread, contact surface area and similar measures. While these are certainly important factors to predict the robustness of a grasp, they do not carry information on the feasibility of the reaching action needed to execute the grasp. Based on our observations of physical pick-up experiments, we suggest that the execution success of a pick-up task is partially dependant on the easiness of the reaching movement. We present our metric, which combines 2 measures involving arm-kinematics and an existing hand heuristic metric. Results of simulated as well as physical experiments in our robot, Crichton, are presented.

Robot path planning in human environments benefits significantly from considering more than obstacle avoidance, and recent works in this area proposed safety and comfort considerations. One shortcoming of current approaches is that humans' behavior is modeled as independent of robot's motions. In this work, we aim to give this anticipation ability to a robot by simulating people's reaction to robot's motion during planning. Our approach is based on extracting a static plan using A* search on the grid map by minimizing safety, disturbance and path length costs and then refining it by simulating humans' reaction using the Social Force Model. With two example scenarios in simulation and two on the real system, we provide qualitative examination of the resulting robot paths and demonstrate that robots can exhibit social behaviors that is not possible to model with standard approaches. This work serves as a primer for quantitative user studies, and we hope will urge future robot path planners to consider a richer set of social capabilities.

This work introduces an affordance characterization employing mechanical wrenches as a metric for predicting and planning with workspace affordances. Although affordances are a commonly used high-level paradigm for robotic task-level planning and learning, the literature has been sparse regarding how to characterize the agent in this object-agent-environment framework. In this work, we propose decomposing a behavior into a vocabulary of characteristic requirements and capabilities that are suitable to predict the affordances of various parts of the workspace. Specifically, we investigate mechanical wrenches as a viable representation of these affordance requirements and capabilities. We then use this vocabulary in a planning system to compose complex motions from simple behavior types in continuous space. The utility of the framework for complex planning is demonstrated on example scenarios both in simulation and with real-world industrial manipulators.

In the United States, there has been a diverse set of robotics programs funded by different agencies 2007. Formulation of a road map for robotics in the United States to establish the basis for a coordinated strategy has been discussed. Four different workshops have been arranged: (1) manufacturing and logistics, (2) health-care and medical robotics, (3) service robotics, and (4) emerging technologies.

We present a 3D model-based visual tracking approach using edge and keypoint features in a particle filtering framework. Recently, particle-filtering-based approaches have been proposed to integrate multiple pose hypotheses and have shown good performance, but most of the work has made an assumption that an initial pose is given. To ameliorate this limitation, we employ keypoint features for initialization of the filter. Given 2D–3D keypoint correspondences, we randomly choose a set of minimum correspondences to calculate a set of possible pose hypotheses. Based on the inlier ratio of correspondences, the set of poses are drawn to initialize particles. After the initialization, edge points are employed to estimate inter-frame motions. While we follow a standard edge-based tracking, we perform a refinement process to improve the edge correspondences between sampled model edge points and image edge points. For better tracking performance, we employ a first-order autoregressive state dynamics, which propagates particles more effectively than Gaussian random walk models. The proposed system re-initializes particles by itself when the tracked object goes out of the field of view or is occluded. The robustness and accuracy of our approach is demonstrated using comparative experiments on synthetic and real image sequences.

In this article we propose a framework for performing embodied evolution with a limited number of robots, by utilizing time-sharing in subpopulations of virtual agents hosted in each robot. Within this framework, we explore the combination of within-generation learning of basic survival behaviors by reinforcement learning, and evolutionary adaptations over the generations of the basic behavior selection policy, the reward functions, and metaparameters for reinforcement learning. We apply a biologically inspired selection scheme, in which there is no explicit communication of the individuals’ fitness information. The individuals can only reproduce offspring by mating—a pair-wise exchange of genotypes—and the probability that an individual reproduces offspring in its own subpopulation is dependent on the individual’s ‘‘health,’’ that is, energy level, at the mating occasion. We validate the proposed method by comparing it with evolution using standard centralized selection, in simulation, and by transferring the obtained solutions to hardware using two real robots.

In this paper, we describe a concise method for the feature-based representation of regions in an indoor environment and show how it can also be applied for door-passage-independent detection of transitions between regions to improve communication with a human user.

This paper presents a distributed approach to enabling mobile robot swarms to track multiple targets moving unpredictably. The proposed approach consists of two constituent algorithms: local interaction and target tracking. When the robots are faster than the targets, Lyapunov theory can be applied to show that the robots converge asymptotically to each vertex of the desired equilateral triangular configurations while tracking the targets. Toward practical implementation of the algorithms, it is important to realize the observation capability of individual robots in an inexpensive and efficient way. A new proximity sensor that we call dual rotating infrared (DRIr) sensor is developed to meet these requirements. Both our simulation and experimental results employing the proposed algorithms and DRIr sensors confirm that the proposed distributed multi-target tracking method for a swarm of robots is effective and easy to implement.

Based on concepts of the human visual system, computational visual attention systems aim to detect regions of interest in images. Psychologists, neurobiologists, and computer scientists have investigated visual attention thoroughly during the last decades and profited considerably from each other. However, the interdisciplinarity of the topic holds not only benefits but also difficulties: Concepts of other fields are usually hard to access due to differences in vocabulary and lack of knowledge of the relevant literature. This article aims to bridge this gap and bring together concepts and ideas from the different research areas. It provides an extensive survey of the grounding psychological and biological research on visual attention as well as the current state of the art of computational systems. Furthermore, it presents a broad range of applications of computational attention systems in fields like computer vision, cognitive systems, and mobile robotics. We conclude with a discussion on the limitations and open questions in the field.

In this paper we present a design study and technical specifications of a high performance robotic manipulator to be used for ball catching experiments using commercial off-the-shelf (COTS) components. Early evaluation shows that very good performance can be achieved using standardized PowerCube actuator modules from Amtec and a standard workstation using CAN bus communication. Implementation issues of low-level control and software platform are also described, as well as early experimental evaluation of the system.

This paper presents practical design and hardware implementation issues of self-configuring swarms of autonomous mobile robots. For the purpose, we develop a new low-cost position detection system that we call dual rotating infrared (DRIr) sensor. The DRIr sensor can provide robots with advanced sensing capabilities that give reliable information about the position and surface geometry of neighboring robots and obstacles. Special focus is placed on how to realize the observation and object identification of mobile robots through the use of DRIr sensors. We verify the functionality and performance of the DRIr sensors mounted on a commercial mobile robot. Experimental results show that a swarm of mobile robots equipped with the DRIr sensors can autonomously configure themselves into an area.

We address the problem of a swarm of autonomous mobile robotic sensors generating geometric shapes to build wireless ad hoc surveillance sensor networks. Robot swarms with limited observation are required to form different shapes under different task conditions. To do this, we propose decentralized coordination enabling a robot swarm dispersed across an area to form a desired shape. Our approach has emphn robots generate a circumscribed circle of a regular emphn-polygon based on local interaction with neighboring robots. The approach also enables a large robot swarm to form concentric circles through consensus. We mathematically demonstrate convergence confirming the feasibility using extensive simulation. Our results indicate that our approach is applicable to mobile sensor network surveillance and security networks.

We address the problem of dispersing a large number of autonomous mobile robots toward building wireless ad hoc sensor networks performing environmental monitoring and control. For the purpose, we propose the adaptive triangular mesh generation algorithm that enables robots to generate triangular meshes of various sizes adapting to changing environmental conditions. A locally interacting, geometric technique allows robots to generate each triangular mesh with their two neighbor robots. Specifically, we have assumed that robots are not allowed to have the identifier, any pre-determined leaders or common coordinate systems, and any explicit communication. Under such minimal conditions, the positions of the robots were shown to converge to the desired distribution, which was mathematically proven and also verified through extensive simulations. Our preliminary results indicate that the proposed algorithm can be applied to the problem regarding the coverage of an area of interest by a swarm of mobile sensors.

In this article, we explore an evolutionary approach to the optimization of potential-based shaping rewards and meta-parameters in reinforcement learning. Shaping rewards is a frequently used approach to increase the learning performance of reinforcement learning, with regards to both initial performance and convergence speed. Shaping rewards provide additional knowledge to the agent in the form of richer reward signals, which guide learning to high-rewarding states. Reinforcement learning depends critically on a few meta-parameters that modulate the learning updates or the exploration of the environment, such as the learning rate α, the discount factor of future rewards γ, and the temperature τ that controls the trade-off between exploration and exploitation in softmax action selection. We validate the proposed approach in simulation using the mountain-car task. We also transfer shaping rewards and meta-parameters, evolutionarily obtained in simulation, to hardware, using a robotic foraging task.

An annotated data set is presented meant to help researchers in developing, evaluating, and comparing various approaches in robotics for building space representations appropriate for communicating with humans. The data consist of omnidirectional images, laser range scans, sonar readings, and robot odometry. A set of base-level human spatial concepts is used to annotate the data.

Autonomous navigation in urban environments inevitably leads to having to switch between various, sometimes conflicting control tasks. Sting Racing, a collaboration between Georgia Tech and SAIC, has developed a modular control architecture for this purpose and thispaperdescribestheoperationanddefinitionofthisarchitecturethroughso-callednested hybrid automata. We show how to map the requirements associated with the DARPA Urban GrandChallengeontothesenestedautomataandillustratetheiroperationthroughanumber of experimental results.

Hierarchical reinforcement learning (RL) algorithms can learn a policy faster than standard RL algorithms. However, the applicability of hierarchical RL algorithms is limited by the fact that the task decomposition has to be performed in advance by the human designer. We propose a Lamarckian evolutionary approach for automatic development of the learning structure in hierarchical RL. The proposed method combines the MAXQ hierarchical RL method and genetic programming (GP). In the MAXQ framework, a subtask can optimize the policy independently of its parent task's policy, which makes it possible to reuse learned policies of the subtasks. In the proposed method, the MAXQ method learns the policy based on the task hierarchies obtained by GP, while the GP explores the appropriate hierarchies using the result of the MAXQ method. To show the validity of the proposed method, we have performed simulation experiments for a foraging task in three different environmental settings. The results show strong interconnection between the obtained learning structures and the given task environments. The main conclusion of the experiments is that the GP can find a minimal strategy, i.e., a hierarchy that minimizes the number of primitive subtasks that can be executed for each type of situation. The experimental results for the most challenging environment also show that the policies of the subtasks can continue to improve, even after the structure of the hierarchy has been evolutionary stabilized, as an effect of Lamarckian mechanisms

Application of SLAM outdoors is challenged by complexity, handling of non-linearities and flexible integration of a diverse set of features. A graphical approach to SLAM is introduced that enables ...

We demonstrate that millimeter-level manipulation accuracy can be achieved without the static camera registration typically required for visual servoing. We register the camera online, converging in seconds, by visually tracking features on the robot and filtering the result. This online registration handles cases such as perturbed camera positions, wear and tear on camera mounts, and even a camera held by a human. We implement the approach on a Schunk LWA4 manipulator and Logitech C920 camera, servoing to target and pre-grasp configurations. Our filtering software is available under a permissive license (Software available at http://github.com/golems/reflex).

We present a system in which a flexible team of robots coordinates to assemble large, complex, and diverse structures autonomously. Our system operates across a wide range of spatial scales and tolerances, using a hierarchical perception architecture. For the successful execution of very precise assembly operations under initial uncertainty, our system starts with high-field of view but low accuracy sensors, and gradually uses low field-of-view but high accuracy sensors. Our system also uses a failure detection and recovery system, integrated with this hierarchical perception architecture: upon losing track of a feature, our system retracts to using high-field of view systems to re-localize. Additionally, we contribute manipulation skills and tools necessary to assemble large structures with high precision. First, the team of robots coordinates to transport large assembly parts which are too heavy for a single robot to carry. Second, we develop a new tool which is capable of co-localizing holes and fasteners for robust insertion and fastening. We present real robot experiments where we measure the contribution of the hierarchical perception and failure recovery approach to the robustness of our system. We also present an extensive set of experiments where our robots successfully insert all 80 of the attempted fastener insertion operations.

© The Author(s) 2015. In this paper we present algorithms and experiments for multi-scale assembly of complex structures by multi-robot teams. We also focus on tasks where successful completion requires multiple types of assembly operations with a range of precision requirements. We develop a hierarchical planning approach to multi-scale perception in support of multi-scale manipulation, in which the resolution of the perception operation is matched with the required resolution for the manipulation operation. We demonstrate these techniques in the context of a multi-step task where robots assemble large box-like objects, inspired by the assembly of an airplane wing. The robots begin by transporting a wing panel, a coarse manipulation operation that requires a wide field of view, and gradually shifts to a narrower field of view but with more accurate sensors for part alignment and fastener insertion. Within this framework we also provide for failure detection and recovery: upon losing track of a feature, the robots retract to using wider field of view systems to re-localize. Finally, we contribute collaborative manipulation algorithms for assembling complex large objects. First, the team of robots coordinates to transport large assembly parts which are too heavy for a single robot to carry. Second, the fasteners and parts are co-localized for robust insertion and fastening. We implement these ideas using four KUKA youBot robots and present experiments where our robots successfully complete all 80 of the attempted fastener insertion operations.

Robots should be able to reason about using objects in the environment to assist task completion. If faced with an unexpected situation, such as a critically damaged floor, robots should be able to evaluate different ways to resolve the situation rather than directly declaring failure. In this paper, we present the first online framework that enables robots to reason about using environment objects as tools to overcome obstructions, allowing robots to solve previously unsolvable tasks. The framework utilizes constraint relaxed planning to detect unavoidable obstructions. In case the robot becomes disconnected from the goal by an obstruction, the system guides the robot to search for and utilize an object in the environment to overcome the obstruction using the concept of inverse affordances. We verified the framework in a realistic simulation environment for the PR2 robot.

In this paper we present an approach for creating complete shape representations from a single depth image for robot grasping. We introduce algorithms for completing partial point clouds based on the analysis of symmetry and extrusion patterns in observed shapes. Identified patterns are used to generate a complete mesh of the object, which is, in turn, used for grasp planning. The approach allows robots to predict the shape of objects and include invisible regions into the grasp planning step. We show that the identification of shape patterns, such as extrusions, can be used for fast generation and optimization of grasps. Finally, we present experiments performed with our humanoid robot executing pick-up tasks based on single depth images and discuss the applications and shortcomings of our approach.

In this paper, we present an information-based approach to select a reduced number of landmarks and poses for a robot to localize itself and simultaneously build an accurate map. We develop an information theoretic algorithm to efficiently reduce the number of landmarks and poses in a SLAM estimate without compromising the accuracy of the estimated trajectory. We also propose an incremental version of the reduction algorithm which can be used in SLAM framework resulting in information based reduced landmark SLAM. The results of reduced landmark based SLAM algorithm are shown on Victoria park dataset and a Synthetic dataset and are compared with standard graph SLAM (SAM [6]) algorithm. We demonstrate a reduction of 40–50% in the number of landmarks and around 55% in the number of poses with minimal estimation error as compared to standard SLAM algorithm.

For a mobile manipulator to interact with large everyday objects, such as office tables, it is often important to have dynamic models of these objects. However, as it is infeasible to provide the robot with models for every possible object it may encounter, it is desirable that the robot can identify common object models autonomously. Existing methods for addressing this challenge are limited by being either purely kinematic, or inefficient due to a lack of physical structure. In this paper, we present a physics-based method for estimating the dynamics of common non-holonomic objects using a mobile manipulator, and demonstrate its efficiency compared to existing approaches.