Search

The feasibility of using chip-based RFID designs as wireless sensor tags open a wide range of application possibilities in the field of robotics. However, multi-step lithography manufacturing and/or MEMS techniques are often required for the industrial-grade fabrication of such sensors. In this paper, we present a simple, home-based, two-step fabrication process to produce chipless RF-based wireless sensors. We use an office-based inkjet printer to produce the antenna traces using silver conductive ink. Kirigami-inspired designs are used to produce four sensors responsive to various mechanical stimuli commonly used for DIY robotic projects (contact, compression, extension, and bend). We demonstrate sensing and wireless transmission of the detected mechanical stimuli through the proposed chipless sensor tags with reliable consistency. The tags can be replicated quickly with the inkjet-printing method. This paper also contains analyses on the effects of varying the dimensions and electrical parameters of the tags. The developed antennas in this work can be used as wireless mechano-responsive sensors for robotic applications.

This paper presents a new open-source mechanical design of a 6-DOF anthropomorphic ALARIS robotic hand that can serve as a low-cost design platform for further customization and utilization for research and educational purposes. The presented hand design employs linkage-based three-phalange finger and two-phalange adaptive thumb designs with non-backdrivable worm-and-rack transmission mechanisms. Combination of design improvements and solutions, discussed in the paper, are implemented in a functional robotic hand prototype with powerful grasping capabilities, which utilizes off-the-shelf inexpensive components and 3D printing technology ensuring the hand low manufacturing cost and replicability. The open-source mechanical design of the presented ALARIS robotic hand is freely available for downloading from the authors’ research lab web-site https://www.alaris.kz and https://github.com/alarisnu/alaris_hand.

This paper proposes a drive-through recharging strategy for a quadrotor. First, a ground charging station is constructed with a portable charging wire, then with two conductive hooks connected to the battery, the quadrotor is flying through the station. Finally, after connecting the hooks to the charging wire, the quadrotor can be recharged without landing or stopping, similar to a drive-through. During the charging, the quadrotor is still able to accomplish local monitoring assignments. The main challenge of this work is to ensure that the hooks successfully connect to the charging wire. Though visual-based techniques appear promising to handle this problem, the detection of a thin charging wire is challenging especially during the flight. Besides, the conductive hooks underneath the quadrotor are out of the camera view, which brings more challenges to the problem. Therefore, in this paper, the aiming problem is directed into the position constraint control problem of the quadrotor. Specifically, a virtual aiming box is considered near the charging wire center, then a control design is proposed to constrain the position of the quadrotor to ensure the hooks position always within the virtual aiming box. Moreover, to enhance the system robustness against the outdoor environment, a wind disturbance estimator is proposed in the control design. Both theoretical analysis and experimental validation are carried out to demonstrate the effectiveness of the proposed strategy.

This paper presents a novel approach that allows a swarm of heterogeneous robots to produce simultaneously segregative and flocking behaviors using only local sensing. These behaviors have been widely studied in swarm robotics and their combination allows the execution of several complex tasks, ranging from surveillance and reconnaissance, to search and rescue, to transport, and to foraging. Although there are several works in the literature proposing different strategies to achieve these behaviors, to the best of our knowledge, this paper is the first to propose an algorithm that emerges simultaneously behaviors and do not rely on global information or communication. Our approach consists of modeling the swarm as a Gibbs Random Field (GRF) and using appropriate potential functions to reach segregation, cohesion and consensus on the velocity of the swarm. Simulations and proof-of-concept experiments using real robots are presented to evaluate the performance of our methodology in comparison to some of the state-of-the-art works that tackle segregative behaviors., Comment: 7 pages, 11 figures, accepted by ICRA 2021

Several applications of video rely on camera calibration, a key enabler towards the measurement of metric parameters from images. Existing calibration methods have not been adapted for use with cameras that are hard to reach such as security cameras. This paper presents a drone-based camera calibration technique for the calibration of cameras in a broader range of operating environments. The technique is enabled by the use of a drone that is equipped with location sensors. In the proposed approach, the drone is used to sample points in the 3D space and its detection on the images provides the 2D matching points enabling the calibration. To design a flight path that allows for an accurate calibration, the paper proposes a methodology to evaluate the impact of path parameters and drone localization and detection uncertainties on the calibration uncertainty. This methodology is applied to evaluate the performance of different sampling paths for the calibration of a large diversity of cameras for the purpose of recommending reliable path parameters.

Regular irradiation of indoor environments with ultraviolet C (UVC) light has become a regular task for many indoor settings as a result of COVID-19, but current robotic systems attempting to automate it suffer from high costs and inefficient irradiation. In this paper, we propose a purpose-made inexpensive robotic platform with off-the-shelf components and standard navigation software that, with a novel algorithm for finding optimal irradiation locations, addresses both shortcomings to offer affordable and efficient solutions for UVC irradiation. We demonstrate in simulations the efficacy of the algorithm and show a prototypical run of the autonomous integrated robotic system in an indoor environment. In our sample instances, our proposed algorithm reduces the time needed by roughly 30\% while it increases the coverage by a factor of 35\% (when compared to the best possible placement of a static light)., Preliminary version of this paper will be published in the ICRA 2021 conference

Most existing surgical robots employ straight rigid instruments and are not compatible with imaging modalities, especially magnetic resonance imaging (MRI) that presents restrictive constraints on the robot and actuator materials. Employing continuum distal end effector and fulfilling multi-imager compatibility will potentially lead to wide adoption of surgical robots in intraoperative image-guided minimally invasive surgery (MIS). This paper introduces a 3-dimensional (3D) printed polyamide continuum robot with 2-degree of freedom, driven by modular shape memory alloy (SMA) spring actuators that enable real-time distal manipulation in 3D workspace. The multi-imager compatibility is conditionally satisfied by utilizing no ferromagnetic materials and MRI-conditional actuators in the robotic system. The use of modular SMA allows repeatable configuration setting, easy integration with active cooling strategies, and facilitates the creation of a sterile barrier between the end effector module and the actuation module. Detailed design of the robot, kinematics modeling and actuator modeling are discussed in the paper. We experimentally verified the robot kinematics and evaluated the dynamic performance of the continuum robot. It features an operating bandwidth of 0.12 Hz at -3 dB and a root-mean-square error of 0.98 mm under model predictive control when tracking sinusoidal signals, both with ±10 mm amplitude (distal bending angle of ±80◦). As a demonstration, the robot was used as a flexible endoscope manipulator and steered under inputs from a joystick to show its real-time performance.

In modern agriculture, measuring phenotypic traits helps breeders monitor plant growth, increase yield, and provide food, feed, and fiber. Traditional phenotyping requires intensive manual work, partially being intrusive. In this paper, we investigate the challenge of measuring phenotypic traits in an automated fashion through mobile robots operating in field environments. In particular, we want to measure plants from images acquired by mobile robots instead of using data from a static scanning environment. We propose to use a differentiable rendering approach to deform a generic 3D template of a plant to fit the observation recorded by a robot while ensuring a coherent deformation of the plant template. The experiments presented in this paper suggest that our approach allows for 3D reconstruction of different plant species at different growth stages using single images. From that model, we can compute important phenotypic traits, such as the leaf area index.

Scene flow represents the motion of points in the 3D space, which is the counterpart of the optical flow that represents the motion of pixels in the 2D image. However, it is difficult to obtain the ground truth of scene flow in the real scenes, and recent studies are based on synthetic data for training. Therefore, how to train a scene flow network with unsupervised methods based on real-world data shows crucial significance. A novel unsupervised learning method for scene flow is proposed in this paper, which utilizes the images of two consecutive frames taken by monocular camera without the ground truth of scene flow for training. Our method realizes the goal that training scene flow network with real-world data, which bridges the gap between training data and test data and broadens the scope of available data for training. Unsupervised learning of scene flow in this paper mainly consists of two parts: (i) depth estimation and camera pose estimation, and (ii) scene flow estimation based on four different loss functions. Depth estimation and camera pose estimation obtain the depth maps and camera pose between two consecutive frames, which provide further information for the next scene flow estimation. After that, we used depth consistency loss, dynamic-static consistency loss, Chamfer loss, and Laplacian regularization loss to carry out unsupervised training of the scene flow network. To our knowledge, this is the first paper that realizes the unsupervised learning of 3D scene flow from monocular camera. The experiment results on KITTI show that our method for unsupervised learning of scene flow meets great performance compared to traditional methods Iterative Closest Point (ICP) and Fast Global Registration (FGR). The source code is available at: https://github.com/IRMVLab/3DUnMonoFlow., ICRA2021

In this paper, we rethink the problem of scene reconstruction from an embodied agent's perspective: While the classic view focuses on the reconstruction accuracy, our new perspective emphasizes the underlying functions and constraints such that the reconstructed scenes provide \em{actionable} information for simulating \em{interactions} with agents. Here, we address this challenging problem by reconstructing an interactive scene using RGB-D data stream, which captures (i) the semantics and geometry of objects and layouts by a 3D volumetric panoptic mapping module, and (ii) object affordance and contextual relations by reasoning over physical common sense among objects, organized by a graph-based scene representation. Crucially, this reconstructed scene replaces the object meshes in the dense panoptic map with part-based articulated CAD models for finer-grained robot interactions. In the experiments, we demonstrate that (i) our panoptic mapping module outperforms previous state-of-the-art methods, (ii) a high-performant physical reasoning procedure that matches, aligns, and replaces objects' meshes with best-fitted CAD models, and (iii) reconstructed scenes are physically plausible and naturally afford actionable interactions; without any manual labeling, they are seamlessly imported to ROS-based simulators and virtual environments for complex robot task executions., Comment: ICRA 2021 paper. Project: https://sites.google.com/view/icra2021-reconstruction

In this paper we discuss how to use a barrier function that is subject to kinematic constraints and limited sensing in order to guarantee that fixed wing unmanned aerial vehicles (UAVs) will maintain safe distances from each other at all times despite being subject to limited range sensing constraints. Prior work has shown that a barrier function can be used to guarantee safe system operation when the state can be sensed at all times. However, in this paper we show that this construction does not guarantee safety when the UAVs are subject to limited range sensing. To resolve this issue, we introduce a method for constructing a new barrier function that accommodates limited sensing range from a previously existing barrier function that may not necessarily accommodate limited range sensing. We show that, under appropriate conditions, the newly constructed barrier function ensures system safety even in the presence of limited range sensing. We demonstrate the contribution of this paper in a scenario of 20 fixed wing aircraft, where because of the proposed algorithm, the vehicles are able to maintain safe distances from each other even though the vehicles are subject to limited range sensing., \c{opyright} 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes,creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works

Vehicle Re-identification (Re-ID) aims to retrieve all instances of query vehicle images present in an image pool. However viewpoint, illumination, and occlusion variations along with subtle differences between two unique images pose a significant challenge towards achieving an effective system. In this paper, we emphasize upon enhancing the performance of visual feature based ReID system by improving feature embedding quality and propose (1) an attention-guided hierarchical feature extractor (HFE) that leverages the structure of a backbone CNN to extract coarse and fine-grained features and (2) to train the proposed network within a hard negative adversarial framework that generates samples exhibiting extreme variations, encouraging the network to extract important distinguishing features across varying scales. To demonstrate the effectiveness of the proposed framework we use VERI-Wild, VRIC and Veri-776 datasets that exhibit extreme intra-class and minute inter-class differences and achieve state-of-the-art (SoTA) performance. Codes related to this paper are publicly available at https://github.com/PS06/VReID.

Due to the high positioning accuracy and relatively low prices, SCARA robots are widely used in industrial fields. The objective of this paper is to propose a hand-eye self-calibration algorithm for SCARA robots which could consider both accuracy and computational cost. The previous global optimal hand-eye calibration algorithms based on branch-and-bound (BnB) optimization is limited by their expensive computational cost. The speed of these algorithms depends on the volume of the search space to a large extent, which is the main concern in this paper. Instead of searching over the 3-dimensional parameter space corresponding to the rotation component of hand-eye pose, a new 2-dimensional search space is defined by separating and coupling some calibration parameters by means of the special structure of SCARA robots, which have 4 degrees of freedom (DoFs) including three translation DoFs and only one rotation DoF. The simulation and real experiments show the similar accuracy but much faster speed of the proposed algorithm compared with previous optimal algorithms based on BnB.

This paper describes the optimization of a neuromorphic architecture for printable organic neurons as part of an ongoing project to develop Form + Function 4-D Printing. The previously proposed architecture prioritizes simplicity and massive redundancy for a printable analog neural network consisting of only one transistor plus memristors for synapses, per neuron, but sacrifices negative synaptic weights. This paper demonstrates an optimization technique to minimize the insertion of inverting amplifiers to realize a minimal approximating set of negative weights. This helps to develop a compact, printable and accurate neuromorphic computer to bring new function to the 3-D printing of form in multi-functional robotic materials. An example robotic skin is developed with the ability to compute the centroid of touch "compiled into the skin" to an average accuracy of 9.19% in comparison to an unconstrained Artificial Neural Network (ANN). The presented soft robotic skin is fabricated by hand using conventional silicon components, but serves as a proof-of-concept for radical new capabilities in Form + Function 4-D Printing.

This paper introduces a real-time dense planar LiDAR SLAM system, named π-LSAM, for the indoor environment. The widely used LiDAR odometry and mapping (LOAM) framework [1] does not include bundle adjustment (BA) and generates a low fidelity tracking pose. This paper seeks to overcome these drawbacks for the indoor environment. Specifically, we use the plane as the landmark, and introduce plane adjustment (PA) as our back-end to jointly optimize planes and keyframe poses. We present the π-factor to significantly reduce the computational complexity of PA. In addition, we introduce an efficient loop detection algorithm based on the RANSAC framework using planes. In the front-end, our algorithm performs global registration in real time. To achieve this performance, we maintain the local-to-global point-to-plane correspondences scan by scan, so that we only need a small local KD-tree to establish the data association between a LiDAR scan and the global planes, rather than a large global KD-tree used in previous works. With this local-to-global data association, our algorithm directly identifies planes in a LiDAR scan, and yields an accurate and globally consistent pose. Experimental results show that our algorithm significantly outperforms the state-of-the-art LOAM variant, LeGO-LOAM [2], and our algorithm achieves real time.

International audience; Demands on leg degrees of freedom and control precision for bipedal robotics are steadily increasing, especially for the tasks involving walking on a rough terrain. In this paper we present an alternative, as well as a working proof-of-concept. Meet gyrubot: a 5-link almost planar bipedal robot with a torso complemented by a nonanthropomorphic stabilization system, capable of blindly walking through uneven areas. Despite being almost planar, the robot does not need any support in the frontal plane! This paper describes the mechanical design and the architecture of the controllers. We also provide the experimental evidence of the ability of gyrubot to navigate across non-flat terrains.

This paper addresses the localization of contacts of an unknown grasped rigid object with its environment, i.e., extrinsic to the robot. We explore the key role that distributed tactile sensing plays in localizing contacts external to the robot, in contrast to the role that aggregated force/torque measurements play in localizing contacts on the robot. When in contact with the environment, an object will move in accordance with the kinematic and possibly frictional constraints imposed by that contact. Small motions of the object, which are observable with tactile sensors, indirectly encode those constraints and the geometry that defines them. We formulate the extrinsic contact sensing problem as a constraint-based estimation. The estimation is subject to the kinematic constraints imposed by the tactile measurements of object motion, as well as the kinematic (e.g., non-penetration) and possibly frictional (e.g., sticking) constraints imposed by rigid-body mechanics. We validate the approach in simulation and with real experiments on the case studies of fixed point and line contacts. This paper discusses the theoretical basis for the value of distributed tactile sensing in contrast to aggregated force/torque measurements. It also provides an estimation framework for localizing environmental contacts with potential impact in contact-rich manipulation scenarios such as assembling or packing., Comment: 6 pages, 8 figures, ICRA 2021

In this paper, we propose an approach for robust visual Simultaneous Localisation and Mapping (SLAM) in underwater environments leveraging acoustic, inertial and altimeter/depth sensors. Underwater visual SLAM is challenging due to factors including poor visibility caused by suspended particles in water, a lack of light and insufficient texture in the scene. Because of this, many state-of-the-art approaches rely on acoustic sensing instead of vision for underwater navigation.Building on the sparse visual SLAM system ORB-SLAM2, this paper proposes to improve the robustness of camera pose estimation in underwater environments by leveraging acoustic odometry, which derives a drifting estimate of the 6-DoF robot pose from fusion of a Doppler Velocity Log (DVL), a gyroscope and an altimeter or depth sensor. Acoustic odometry estimates are used as motion priors and we formulate pose residuals that are integrated within the camera pose tracking, local and global bundle adjustment procedures of ORB-SLAM2.The original design of ORB-SLAM2 supports a single map and it enters relocalisation when tracking is lost. This is a significant problem for scenarios where a robot does a continuous scanning motion without returning to a previously visited location. One of our main contributions is to enable the system to create a new map whenever it encounters a new scene where visual odometry can work. This new map is connected with its predecessor in a common graph using estimates from the proposed acoustic odometry. Experimental results on two underwater vehicles demonstrate the increased robustness of our approach compared to baseline ORB-SLAM2 in both controlled, uncontrolled and field environments.

Visual localization plays an indispensable role in robotics. Both learning and hand-crafted feature based methods for relocalization process keep their effectiveness and weakness. However, current algorithms seldom consider these two kinds of features under one framework. In this paper, focusing on this task, we propose a novel relocalization framework for RGB or RGB-D data source, which is composed of coarse localization process by learning features and pose refinement by hand-crafted features. In particular, coarse stage contains deep point cloud generation and registration. In this stage, instead of regressing camera pose directly, the paper novelly designs a neural network called PGNet to construct sparse point cloud with RGB or RGB-D as inputs. Further more, by means of training set, hand-crafted feature space is established. Based on the obtained camera pose in coarse stage, accurate point-to-point correspondences are set up through searching the space. Then accurate camera pose is obtained by applying RANSAC to correspondences or solving PnP. Finally, experiments on both outdoor and indoor benchmark datasets demonstrate state-of-the-art performance over other existing methods.

The Correlated Team Orienteering Problem (CTOP) is a routing problem where the objective is to determine a set of routes that maximizes the summation of collected rewards in the environment while respecting the vehicles’ budget. However, solutions to this problem usually consider static instances and may produce poor results in dynamic real-world scenarios. In this paper, we propose an approach to deal with the execution of missions planned as a CTOP instance, especially when vehicles of the team are prone to failure and may not complete their routes. The main contribution of this paper is a novel anytime heuristic that iteratively adapts the initial set of routes, allowing to increase the overall robustness of the mission and still collect the most profitable rewards. The methodology was thoroughly evaluated considering different scenarios and in all the cases was able to achieve comparable or better results in terms of reward than planning a new set of routes, however, spending considerably less time.

In this paper, a robust RGB-D SLAM system is proposed to utilize the structural information in indoor scenes, allowing for accurate tracking and efficient dense mapping on a CPU. Prior works have used the Manhattan World (MW) assumption to estimate low-drift camera pose, in turn limiting the applications of such systems. This paper, in contrast, proposes a novel approach delivering robust tracking in MW and non-MW environments. We check orthogonal relations between planes to directly detect Manhattan Frames, modeling the scene as a Mixture of Manhattan Frames. For MW scenes, we decouple pose estimation and provide a novel drift-free rotation estimation based on Manhattan Frame observations. For translation estimation in MW scenes and full camera pose estimation in non-MW scenes, we make use of point, line and plane features for robust tracking in challenging scenes. %mapping Additionally, by exploiting plane features detected in each frame, we also propose an efficient surfel-based dense mapping strategy, which divides each image into planar and non-planar regions. Planar surfels are initialized directly from sparse planes in our map while non-planar surfels are built by extracting superpixels. We evaluate our method on public benchmarks for pose estimation, drift and reconstruction accuracy, achieving superior performance compared to other state-of-the-art methods. We will open-source our code in the future.

Myoelectric prosthetic hands are intended to replace the function of the amputee's lost arm. Therefore, developing robotic prosthetics that can mimic not only the appearance and functionality of humans but also characteristics unique to human movements is paramount. Although the impedance model was proposed to realize biomimetic control, this model cannot replicate the characteristics of human movements effectively because the joint angle always converges to the equilibrium position during muscle relaxation. This paper proposes a novel biomimetic control method for myoelectric prosthetic hands integrating the impedance model with the concept of the $\lambda$-type muscle model. The proposed method can dynamically control the joint equilibrium position, according to the state of the muscle, and can maintain the joint angle naturally during muscle relaxation. The effectiveness of the proposed method is evaluated through simulations and a series of experiments on non-amputee participants. The experimental results, based on comparison with the actual human joint angles, suggest that the proposed method has a better correlation with the actual human motion than the conventional methods. Additionally, the control experiments showed that the proposed method could achieve a natural prosthetic hand movement similar to that of a human, thereby allowing voluntary hand opening and closing movements., Comment: This paper is accepted for publication at the International Conference on Robotics and Automation 2021 (ICRA 2021)

This paper describes Resh, a new, statically typed, interpreted programming language and associated runtime for orchestrating multirobot systems. The main features of Resh are: (1) It offloads much of the tedious work of programming such systems away from the programmer and into the language runtime; (2) It is based on a small set of temporal and locational operators; and (3) It is not restricted to specific robot types or tasks. The Resh runtime consists of three engines that collaborate to run a Resh program using the available robots in their current environment. This paper describes both Resh and its runtime and gives examples of its use., Comment: Accepted for publication at ICRA'21

The time-domain passivity approach has been proposed in the literature in a variety of formats to guarantee the stability of teleoperation leader-follower systems. The conventional use of the proposed technique utilizes the control effort at the follower side as the force feedback to be sent back to the user at the leader’s side. However, this has resulted in transparency problems, especially when the follower dynamics are not negligible. On the other hand, four-channel and three-channel Lawrence architectures have been investigated widely in the literature to maximize the transparency of the system when, in most advanced cases, stability is guaranteed using wave-variables. However, wave-variables are historically known for their transparency deterioration problems. In this paper, we propose a two-layer approach taking advantage of the fusion of (a) a more optimal derivation of Lawrence telerobotic architecture (utilizing only two channels), and (b) a two-port time-domain passivity stabilizer while comparing the performance with a one-port passivity stabilizer. The two-channel derivation of the Lawrence architecture allows for implementing a two-port time domain passivity approach, which is investigated in this paper. The performance of this is compared systematically through a multi-objective approach by analyzing dissipated energy besides force and velocity errors for a wide range of time delays and frequencies of excitation. The paper gives a comprehensive view of the efficacy of two-port versus one-port time-domain passivity control when combined with the two-channel derivation of Lawrence architecture.

This paper presents a method for LiDAR sensor-based traversability analysis for autonomous mobile robots in urban environments. Although urban environments are structured environments, a typical terrain comprises hazardous regions for mobile robots. Therefore, a reliable method for detecting traversable regions is required to prevent robots from getting stuck in the middle of the road. Conventional approaches require considerable efforts to obtain a model for traversability analysis for a specific robot or environment. In particular, learning-based methods require explicit training data. This paper introduces a method for traversability mapping based on a self-training algorithm to eliminate the hand labeling process. A neural network was applied to the underlying classifier of the self-training algorithm. With our approach, the model can be learned with even weakly labeled data obtained from robot-specific parameters and the robot’s footprint. In practical experiments, the self-trained model performed better performance than the existing supervised learning method. Moreover, as the fraction of unlabeled data increased, the performance also increased. Therefore, the demonstrations in the urban environments indicate the effectiveness of the proposed method for traversability mapping.

This paper presents a framework that uses language instructions to define the constraints and objectives for robots gathering information about their environment. Designing autonomous robotic sampling missions requires deep knowledge of both autonomy systems and scientific domain expertise. Language commands provide an intuitive interface for operators to give complex instructions to robots. The key insight we leverage is using topological constraints to define routing directions from the language instruction such as ‘route to the left of the island.’ This work introduces three main contributions: a framework to map language instructions to constraints and rewards for robot planners, a topology constrained information gathering algorithm, and an automatic semantic feature detection algorithm for upwelling fronts. Our work improves on existing methods by not requiring training data with language instruction to planner constraint pairs, allowing new robotic domains such as marine robotics to use our method. This paper provides results demonstrating our framework producing correct constraints for 84.6% of instructions, from a systematically generated corpus of over 1.1 million instructions We also demonstrate the framework producing robot plans from language instructions for real-world scientific sampling missions with the Slocum underwater glider.

In this paper, a degeneracy avoidance method for a point and line based visual SLAM algorithm is proposed. Visual SLAM predominantly uses point features. However, point features lack robustness in low texture and illuminance variant environments. Therefore, line features are used to compensate the weaknesses of point features. In addition, point features are poor in representing discernable features for the naked eye, meaning mapped point features cannot be recognized. To overcome the limitations above, line features were actively employed in previous studies. However, since degeneracy arises in the process of using line features, this paper attempts to solve this problem. First, a simple method to identify degenerate lines is presented. In addition, a novel structural constraint is proposed to avoid the degeneracy problem. At last, a point and line based monocular SLAM system using a robust optical-flow based lien tracking method is implemented. The results are verified using experiments with the EuRoC dataset and compared with other state-of-the-art algorithms. It is proven that our method yields more accurate localization as well as mapping results., Comment: 8 pages to be published in ICRA2021

It is widely anticipated that Level-44 (L4) autonomous vehicles - vehicles capable of autonomously performing all driving functions under certain conditions - will eventually become available to the public. However, these vehicles are likely to be expensive due to the additional requirements on hardware, making them potentially inaccessible to the average consumer. This paper introduces the concept of Affordable Autonomy through Cooperative Sensing and Planning (AACSP). This concept enables a vehicle equipped with lower-end sensors such as a Level-26 vehicle - vehicle capable of limited lateral and longitudinal control but requiring a driver to stay engaged - to enter into an autonomous mode whenever it is in proximity with a Level-4 vehicle traveling along the same route. The paper describes the system along with the technical components of cooperative planning & state coordination in detail while giving an overview of the cooperative perception and localization modules that are required to implement the AACSP concept. The paper also presents results from simulation and real vehicles.

Multi-instance scenes are especially challenging for end-to-end visuomotor (image-to-control) learning algorithms. "Pipeline" visual servo control algorithms use separate detection, selection and servo stages, allowing algorithms to focus on a single object instance during servo control. End-to-end systems do not have separate detection and selection stages and need to address the visual ambiguities introduced by the presence of an arbitrary number of visually identical or similar objects during servo control. However, end-to-end schemes avoid embedding errors from detection and selection stages in the servo control behaviour, are more dynamically robust to changing scenes and are algorithmically simpler. In this paper, we present a reactive real-time end-to-end visuomotor learning algorithm for multi-instance reaching. The proposed algorithm uses a monocular RGB image and the manipulator’s joint angles as the input to a light-weight fully-convolutional network (FCN) to generate control candidates. A key innovation of the proposed method is identifying the optimal control candidate by regressing a control-Lyapunov function (cLf) value. The multi-instance capability emerges naturally from the stability analysis associated with the cLf formulation. We demonstrate the proposed algorithm effectively reaching and grasping objects from different categories on a table-top amid other instances and distractors from an over-the-shoulder monocular RGB camera. The network is able to run up to ∼160 fps during inference on one GTX 1080 Ti GPU.

Addressing on monocular visual odometry problem, this paper presents a novel end-to-end network for estimation of camera ego-motion. The network learns the latent space of optical flow (OF) and models sequential dynamics so that the motion estimation is constrained by the relations between sequential images. We compute the OF field of consecutive images and extract the latent OF representation in a self-encoding manner. A Recurrent Neural Network is then followed to examine the OF changes, i.e., to conduct sequential learning. The extracted sequential OF latent space is used to compute the regression of the 6-dimensional pose vector. Particularly, we separately train the encoder in an unsupervised manner. By this means, we avoid non-convergence during the training of the whole network and allow more generalized and effective feature representation. Substantial experiments have been conducted on KITTI and Malaga datasets, and the results demonstrate that our model outperforms most learning-based VO approaches.

This paper introduces a general state estimation framework fusing multiple sensor information for hybrid wheeled-legged robots performing mobile manipulation tasks. At the core of the state estimator is a novel unified odometry for hybrid locomotion which can seamlessly maintain tracking and has no need to switch between stepping and rolling modes. To the best of our knowledge, the proposed odometry is the first work in this area. It is calculated based on the robot kinematics and instantaneous contact points of wheels with sensor inputs from IMU, joint encoders, joint torque sensors estimating wheel contact status, as well as RGB-D camera detecting geometric features of the terrain (e.g. elevation and surface normal vector). Subsequently, the odometry output is utilized as the motion model of a 3D Lidar map-based Monte Carlo Localization module for drift-free state estimation. As part of the framework, visual localization is integrated to provide high precision guidance for the robot movement relative to an object of interest. The proposed approach was verified thoroughly by two experiments conducted on the Pholus robot with OptiTrack measurements as ground truth.

Three-dimensional reconstruction in confined spaces is important for the manufacturing of aircraft wings, inspection of narrow pipes, examination of turbine blades, etc. It is also challenging because confined spaces tend to lack a positioning infrastructure, and conventional sensors often cannot detect objects in close range. Therefore, such tasks require a sensor that is compact, operates in short-range, and able to localize itself. In this paper, we introduce a miniature and low-cost 3D scanning system including an active laser-stripe triangulation hardware, integrated inertial sensors, and a Simultaneous Localization and Mapping (SLAM) software tailored for the sensor. The proposed system is capable of reconstructing photo-realistic 3D point cloud in real-time in spite of its compact monocular configuration. To achieve this capability, we propose an approach to capture both color and geometry using alternating shutter-speed on a single camera. A novel SLAM method is proposed to accurately localize the sensor by fusing laser, camera, and inertial measurements. Evaluation of localization accuracy and comparison on reconstruction performance against a significantly larger commercial off-the-shelf sensor demonstrate the proposed system’s advantages in real-world applications.

In this paper, we propose a vanishing point aided LiDAR-Visual-Inertial estimator to achieve real-time, low-drift and robust pose estimation. The proposed method is mainly composed of 3 sequential modules, namely IMU-aided vanishing point (VP) detection module, voxel-map based feature depth association module, and visual inertial fixed-lag smoother module. The IMU-aided VP detection module will detect feature points, line segments and vanishing points to establish robust correspondences in successive frames. In particular, we propose to use 1-line RANSAC method to provide stable VP hypotheses and polar grid to accelerate vanishing point hypothesis validation. After that, we propose a novel voxel-map based feature depth association method, to retrieve depth and assign depth to visual feature efficiently. Finally, the visual inertial fixed-lag smoother module is proposed to jointly minimize error terms. Experiments show that our method outperforms the state-of-the-art visual-inertial odometry and LiDAR-visual estimator in both indoor and outdoor environments.

The inner/low-level control loop of most industrial robotic manipulators is protected from any modification by a closed control architecture. The only way to specify joint inputs to them is through position or velocity commands. Furthermore, the inner controller is unknown/uncertain, as it is not revealed to the user. This makes it very difficult to determine the configuration of the inner controller, including the structure and values of the control gains. As a result, integrating external sensory feedback systems into a closed architecture system becomes a difficult task because the interaction of the external feedback loop with the inner control loop can affect the stability of the overall system. In this paper, a stable control strategy is proposed to generate the joint velocity commands for industrial robotic manipulators with uncertain closed control architecture. Based on the proposed method, external feedback controllers can be added regardless of the inner control loop configuration. The proposed external (or outer) control loop approach provides a high degree of design flexibility by enabling smooth implementation of various modern control applications on industrial robots. Unlike previous studies, the proposed control method is not limited to a particular configuration of the controller in the inner loop, nor to the structure of its control gains. The proposed controller design is validated on the UR 5e industrial manipulator.

A two-wheeled self-balancing robot is a statically unstable non-linear system with strong coupling dynamics. Common practices in the development of control systems for such robots are either to linearise the region of application to be used with linear controllers or to use complex nonlinear controllers such as Fuzzy logic, Sliding Mode, and Neural Networks. Nonetheless, in this paper, we are proposing a novel to this field concept of switching control that would adjust its approach depending on the evaluation of the current states. The performance of the proposed controller was assessed against exemplary solely linear and solely non-linear controllers in simulated tests. The tested were evaluated against dynamic criteria (distance traveled, max. angular deviation, etc.), control criteria (settling time, % overshoot, etc.), and environmental criterion of energy consumption. The results showed an interesting behavior of the proposed controller, with superior performance in many cases.

Active wireless capsule endoscopy (WCE) under magnetic actuation is a promising technology to reduce the inspection time and relieve the burden of physicians. In this paper, we propose a reciprocally rotating magnetic actuation method for trajectory following of a capsule and develop its dynamic model. For the trajectory following task, we investigate the closed-loop tracking control strategies based on different controllers to actuate the capsule in the complex environments. The effectiveness of our method is validated in extensive experiments in a simulation environment as well as in an ex-vivo pig colon. The results demonstrate that the proposed method can accurately and efficiently actuate the capsule to follow the desired trajectory in the complex environments, achieving tracking errors on the order of millimeter. Moreover, the experiments on the ex-vivo pig colon show that the proposed reciprocally rotating magnetic actuation method has the potential to reduce the clinical risks and improve the safety and clinical acceptability of this technology.

This paper describes a novel framework for autonomous exploration in large and complex environments. We show that the framework is efficient as a result of its hierarchical structure, where at one level it maintains a sparse representation of the environment and at another level, a dense representation is used within a local planning horizon around the robot. The exploration path is computed at the two levels, coarsely at the global scale and finely around the robot. Such a framework produces detailed paths in the vicinity of the robot, while trades off data resolution far away from the robot for computational efficiency. In experiments, we evaluate our method with a real robot exploring large and complex indoor and outdoor environments. Results show that our method is twice as efficient in covering spaces while using less than one-fifth of processing in comparison to state-of-the-art methods.

Lecturers of Engineering courses around the world are struggling to increase the engagement of students through the introduction of appropriate hands-on activities and assignments. In Biomechatronics and Robotics courses these assignments typically focus on how certain devices are designed, modelled, fabricated, or controlled. The hardware for these assignments is usually purchased by some external vendor and the students only get the chance to analyze it or program it, so as to execute a useful task (e.g., programming mobile robots to perform path following tasks). Student engagement can be increased by instructing the students to prepare the hardware for their assignment. This also increases the sense of ownership of the project outcomes. In this paper, we present how a robotic gripper / hand design project and the introduction of a grasping and manipulation competition as a course assignment, can significantly increase the student engagement and their understanding of the taught concepts. The presented best practices have been trialed over the last four years in two different courses (one undergraduate and one postgraduate) of the Department of Mechanical Engineering at the University of Auckland in New Zealand. For the particular assignment the students were asked to fully develop a robotic gripper or hand from scratch using a single actuator (only the actuator and the power electronics were provided). The performance of the developed devices was assessed through the participation in a grasping and manipulation competition. All the details of the proposed assignment are presented, hoping that they could help other lecturers and teachers to prepare similar activities.

Exploration in environments with sparse rewards remains a challenging problem in Deep Reinforcement Learning (DRL). For the off-policy method, it usually needs a large number of training samples. With the growing dimensions of state and action space, this method becomes more and more sample-inefficient. In this paper, we propose a novel fast exploration method for off-policy reinforcement learning, called Density-based Intrinsic Motivation and Self-adaptive Action Noise (DIMSAN). Our main contribution is twofold: (1) We propose a Density-based Intrinsic Motivation (DIM) method. It introduces a new intrinsic-reward generation mechanism based on samples’ density estimation during experience replay and encourages the agent to seek novel and unfamiliar states. (2) We propose a Self-adaptive Action Noise (SAN) to deal with the exploration-exploitation tradeoffs, which could automatically change the exploration step through adding adaptive action space noise. The synergy between DIM and SAN could guide the agent to search the state and action space with high efficiency. We evaluate our method on the benchmark manipulation tasks and the designed challenging ones. Empirical results show that our method outperforms the existing methods in terms of convergence speed and sample efficiency, especially in challenging tasks.

Recently proposed DNN-based stereo matching methods that learn priors directly from data are known to suffer a drastic drop in accuracy in new environments. Although supervised approaches with ground truth disparity maps often work well, collecting them in each deployment environment is cumbersome and costly. For this reason, many unsupervised domain adaptation methods based on image-to-image translation have been proposed, but these methods do not preserve the geometric structure of a stereo image pair because the image-to-image translation is applied to each view separately. To address this problem, in this paper, we propose an attention mechanism that aggregates features in the left and right views, called Stereoscopic Cross Attention (SCA). Incorporating SCA to an image-to-image translation network makes it possible to preserve the geometric structure of a stereo image pair in the process of the image-to-image translation. We empirically demonstrate the effectiveness of the proposed unsupervised domain adaptation based on the image-to-image translation with SCA., Comment: Accepted to ICRA 2021

This paper presents an optimal planning/guidance approach for a UAV point-to-point navigation in 2D, under wind influence, with the imposition of a general desired airspeed profile. A cost function weighting the travel time and the control effort is minimized through the Pontryagin’s Minimum Principle. An analytical expression for the optimal heading to assure minimum-time flight is also derived. A general wind case in constrained path navigation is solved through an iterative procedure, using artificial potential fields.

We are interested in learning models of non-stationary environments, which can be framed as a multi-task learning problem. Model-free reinforcement learning algorithms can achieve good asymptotic performance in multi-task learning at a cost of extensive sampling, due to their approach, which requires learning from scratch. While model-based approaches are among the most data efficient learning algorithms, they still struggle with complex tasks and model uncertainties. Meta-reinforcement learning addresses the efficiency and generalization challenges on multi task learning by quickly leveraging the meta-prior policy for a new task. In this paper, we propose a meta-reinforcement learning approach to learn the dynamic model of a non-stationary environment to be used for meta-policy optimization later. Due to the sample efficiency of model-based learning methods, we are able to simultaneously train both the meta-model of the non-stationary environment and the meta-policy until dynamic model convergence. Then, the meta-learned dynamic model of the environment will generate simulated data for meta-policy optimization. Our experiment demonstrates that our proposed method can meta-learn the policy in a non-stationary environment with the data efficiency of model-based learning approaches while achieving the high asymptotic performance of model-free meta-reinforcement learning., Comment: 8 pages, 4 figures

Traditional convolution for capturing local structures and relationships remains a key technical limit in 3D semantic segmentation, which neglects the certain influence of the adjacent points on the central point in the disordered local point clouds. In this paper, we propose a novel joint-edge graph convolution neural network (JEGCN), which can extract the dynamic features of each local area and transfer the edge information between the vertex pairs to the adjacent vertices. In the proposed graph convolution module, the adjacent vertices are selected with high classification confidence which can guide the central vertex, and then reweight these vertices. Considering the lack of texture features in 3D point clouds, we incorporate 2D image features to adjacent feature propagation to effectively extract the local and global features of point clouds. The experimental results based on ScanNet and S3DIS datasets demonstrate the effectiveness of the proposed method.

This paper studies the problem of multi-robot cooperative visual-inertial localization where each robot is equipped with only a single camera and IMU. We develop two cooperative visual-inertial odometry (C-VIO) algorithms within the multi-state constraint Kalman filter (MSCKF) framework, in which each robot utilizes not only its own measurements but constraints of common features co-observed with its neighbors within the current sliding window in order to improve the localization accuracy. The first centralized-equivalent algorithm tracks the robot-to-robot cross correlations and prioritizes the pose accuracy while requiring full capacity communication among all the robots during update. The second distributed algorithm ignores the robot-to-robot cross correlations to obtain a scalable, robust and efficient fully distributed structure where each robot only keeps its own states and communicates with its neighbors, while a covariance intersection (CI)-based update strategy is leveraged to guarantee consistency. The proposed algorithms are validated extensively in both Monte-Carlo simulations and real-world datasets, and shown to be able to achieve better accuracy with competitive efficiency.

Depth completion is an important task in computer vision and robotics applications, which aims at predicting accurate dense depth from a single RGB-LiDAR image. Convolutional neural networks (CNNs) have been widely used for depth completion to learn a mapping function from sparse to dense depth. However, recent methods do not exploit any 3D geometric cues during the inference stage and mainly rely on sophisticated CNN architectures. In this paper, we present a cascade and geometrically inspired learning framework for depth completion, consisting of three stages: view extrapolation, stereo matching, and depth refinement. The first stage extrapolates a virtual (right) view using a single RGB (left) and its LiDAR data. We then mimic the binocular stereo-matching, and as a result, explicitly encode geometric constraints during depth completion. This stage augments the final refinement process by providing additional geometric reasoning. We also introduce a distillation framework based on teacher-student strategy to effectively train our network. Knowledge from a teacher model privileged with real stereo pairs is transferred to the student through feature distillation. Experimental results on KITTI depth completion benchmark demonstrate that the proposed method is superior to state-of-the-art methods.

Real-time semantic segmentation is a challenging task as both accuracy and inference speed need to be considered simultaneously. In real-world applications, it is usually achieved by deploying a deep neural network in modern GPU device. However, most of the work focused on real-time semantic segmentation is designed by significantly reducing computation complexity and model size. There are other factors that have a significant impact on inference speed are overlooked, especially when the network is running in modern GPU device. In this paper, we focus on designing a GPU-efficient network as backbone for real-time semantic segmentation. Dense connectivity can preserve and accumulate feature maps of multiple receptive fields and is therefore ideal for semantic segmentation. Therefore, we design a GPU-efficient network (DenseENet) with dense connectivity. The proposed DenseENet shows an obvious advantage in balancing accuracy and inference speed in modern GPU device. Specifically, on Cityscapes test set, DenseENet with a simple FCN decoder achieves 75.2% mIoU with 83.6 FPS for an input of 1024 × 2048 resolution and 73.6% mIoU with 132 FPS for an input of 768 × 1536 resolution on a single GTX 1080Ti card.

International audience; Robotic systems evolving in hazardous and harsh environment are prone to mission failure or system loss in presence of faults. This paper presents a fault tolerant methodology, implemented into a control architecture of an underwater robot that executes biological monitoring missions. High level constraint violations (mission, safety, energy, time and localization) and low level faults (software and hardware faults) are considered using a method based on fault trees. These undesirable events are detected and treated by a fault tolerant module that decides to recover at low level or to give a feedback to the mission manager which selects the high level reaction. This fault tolerant architecture has been tested on real field conditions, and we illustrate our methodology on a set of selected events. We conclude about reliability improvement of low cost underwater robots for complex and long missions.

Recently, tactile sensing has attracted great interest for robotic manipulation. Predicting if a grasp will be stable or not, i.e. if the grasped object will drop out of the gripper while being lifted, can aid robust robotic grasping. Previous methods paid equal attention to all regions of the tactile data matrix or all time-steps in the tactile sequence, which may include irrelevant or redundant information. In this paper, we propose to equip Convolutional Neural Networks with spatial-channel and temporal attention mechanisms (SCT attention CNN) to predict future grasp stability. To the best of our knowledge, this is the first time to use attention mechanisms for predicting grasp stability only relying on tactile information. We implement our experiments with 52 daily objects. Moreover, we compare different spatio-temporal models and attention mechanisms as an empirical study. We found a significant accuracy improvement of up to 5% when using SCT attention. We believe that attention mechanisms can also improve the performance of other tactile learning tasks in the future, such as slip detection and hardness perception.

This paper presents a technique that allows a robot to escort a human to their destination. Unlike tracking where the robot follows the human from behind, the proposed technique locates the robot in front of the human by incorporating human intention in addition to conventional motion prediction. Human head pose is used as an effective past-proven implicit indicator of intention. A particle filter allows accurate estimation and prediction of the non-Gaussian human trajectory. The predicted pose from both the human motion and intention determines the robot control action which leads to efficient autonomous escorting. Experimental analysis shows that the incorporation of the proposed human intention model reduces human position prediction error by approximately 40% when turning. Experimental validation with an omnidirectional mobile robotic platform shows successful and effective escorting compared to the conventional techniques.

This paper considers a multi-target coverage problem where a robot team aims to efficiently cover multi-targets while maintaining connectivity in a distributed manner. A novel knowledge-incorporated policy framework is proposed to derive a distributed, efficient, and connectivity guaranteed coverage policy. In particular, a knowledge-guided policy network (KGPnet) is designed, which consists of observation attention representation, interaction attention representation, and knowledge-guided policy learning. Giving credit to the KGPnet, the connectivity guaranteed coverage policy can be applied to different number targets. Moreover, based on the knowledge of the algebraic connectivity and coverage rate, a comprehensive reward is designed to guide the training of the behavior of multi-target coverage with connectivity maintenance. Furthermore, since the policy learned through deep reinforcement learning (DRL) can not guarantee the connectivity of the robot team, a knowledge-nested policy filtering is designed to filter dis-connectivity policies to satisfy the connectivity constraint based on the knowledge model of connectivity maintenance. Various simulations are conducted to verify the effectiveness of the proposed method. Besides, numerous real-world experiments with three-wheel omnidirectional cars and a motion capture system are presented to demonstrate the practicability of the proposed method.