Search

This paper presents a solution for the autonomous exploration and inspection of Ballast Water Tanks (BWTs) of marine vessels using aerial robots. Ballast tank compartments are critical for a vessel's safety and correspond to confined environments often connected through particularly narrow manholes. The method enables their volumetric exploration combined with visual inspection subject to constraints regarding the viewing distance from a surface. We present evaluation studies in simulation, in a mission consisting of 18 BWT compartments, and in 3 field experiments inside real vessels. The data from one of the experiments is also post-processed to generate semantically-segmented meshes of inspection-important geometries. Geometric models can be associated with onboard camera images for detailed and intuitive analysis., Comment: 8 pages, 7 figures, accepted for publication at the 2023 IEEE International Conference on Advanced Robotics (ICAR)

This paper contributes a novel and modularized learning-based method for aerial robots navigating cluttered environments containing hard-to-perceive thin obstacles without assuming access to a map or the full pose estimation of the robot. The proposed solution builds upon a semantically-enhanced Variational Autoencoder that is trained with both real-world and simulated depth images to compress the input data, while preserving semantically-labeled thin obstacles and handling invalid pixels in the depth sensor's output. This compressed representation, in addition to the robot's partial state involving its linear/angular velocities and its attitude are then utilized to train an uncertainty-aware 3D Collision Prediction Network in simulation to predict collision scores for candidate action sequences in a predefined motion primitives library. A set of simulation and experimental studies in cluttered environments with various sizes and types of obstacles, including multiple hard-to-perceive thin objects, were conducted to evaluate the performance of the proposed method and compare against an end-to-end trained baseline. The results demonstrate the benefits of the proposed semantically-enhanced deep collision prediction for learning-based autonomous navigation., Comment: 8 Pages, 8 figures. Accepted to the IEEE/RSJ International Conference on Intelligent Robots and Systems 2023

This article presents the CERBERUS robotic system-of-systems, which won the DARPA Subterranean Challenge Final Event in 2021. The Subterranean Challenge was organized by DARPA with the vision to facilitate the novel technologies necessary to reliably explore diverse underground environments despite the grueling challenges they present for robotic autonomy. Due to their geometric complexity, degraded perceptual conditions combined with lack of GPS support, austere navigation conditions, and denied communications, subterranean settings render autonomous operations particularly demanding. In response to this challenge, we developed the CERBERUS system which exploits the synergy of legged and flying robots, coupled with robust control especially for overcoming perilous terrain, multi-modal and multi-robot perception for localization and mapping in conditions of sensor degradation, and resilient autonomy through unified exploration path planning and local motion planning that reflects robot-specific limitations. Based on its ability to explore diverse underground environments and its high-level command and control by a single human supervisor, CERBERUS demonstrated efficient exploration, reliable detection of objects of interest, and accurate mapping. In this article, we report results from both the preliminary runs and the final Prize Round of the DARPA Subterranean Challenge, and discuss highlights and challenges faced, alongside lessons learned for the benefit of the community.

This work contributes a marsupial robotic system-of-systems involving a legged and an aerial robot capable of collaborative mapping and exploration path planning that exploits the heterogeneous properties of the two systems and the ability to selectively deploy the aerial system from the ground robot. Exploiting the dexterous locomotion capabilities and long endurance of quadruped robots, the marsupial combination can explore within large-scale and confined environments involving rough terrain. However, as certain types of terrain or vertical geometries can render any ground system unable to continue its exploration, the marsupial system can - when needed - deploy the flying robot which, by exploiting its 3D navigation capabilities, can undertake a focused exploration task within its endurance limitations. Focusing on autonomy, the two systems can co-localize and map together by sharing LiDAR-based maps and plan exploration paths individually, while a tailored graph search onboard the legged robot allows it to identify where and when the ferried aerial platform should be deployed. The system is verified within multiple experimental studies demonstrating the expanded exploration capabilities of the marsupial system-of-systems and facilitating the exploration of otherwise individually unreachable areas., Comment: 6 pages, 6 figures, Submitted to the IEEE/RSJ International Conference on Intelligent Robots and Systems, 2022

This work presents the design, hardware realization, autonomous exploration and object detection capabilities of RMF-Owl, a new collision-tolerant aerial robot tailored for resilient autonomous subterranean exploration. The system is custom built for underground exploration with focus on collision tolerance, resilient autonomy with robust localization and mapping, alongside high-performance exploration path planning in confined, obstacle-filled and topologically complex underground environments. Moreover, RMF-Owl offers the ability to search, detect and locate objects of interest which can be particularly useful in search and rescue missions. A series of results from field experiments are presented in order to demonstrate the system's ability to autonomously explore challenging unknown underground environments., Comment: 8 pages, 9 figures. Submitted to the International Conference on Unmanned Aircraft Systems, 2022

Autonomous exploration of subterranean environments constitutes a major frontier for robotic systems as underground settings present key challenges that can render robot autonomy hard to achieve. This has motivated the DARPA Subterranean Challenge, where teams of robots search for objects of interest in various underground environments. In response, the CERBERUS system-of-systems is presented as a unified strategy towards subterranean exploration using legged and flying robots. As primary robots, ANYmal quadruped systems are deployed considering their endurance and potential to traverse challenging terrain. For aerial robots, both conventional and collision-tolerant multirotors are utilized to explore spaces too narrow or otherwise unreachable by ground systems. Anticipating degraded sensing conditions, a complementary multi-modal sensor fusion approach utilizing camera, LiDAR, and inertial data for resilient robot pose estimation is proposed. Individual robot pose estimates are refined by a centralized multi-robot map optimization approach to improve the reported location accuracy of detected objects of interest in the DARPA-defined coordinate frame. Furthermore, a unified exploration path planning policy is presented to facilitate the autonomous operation of both legged and aerial robots in complex underground networks. Finally, to enable communication between the robots and the base station, CERBERUS utilizes a ground rover with a high-gain antenna and an optical fiber connection to the base station, alongside breadcrumbing of wireless nodes by our legged robots. We report results from the CERBERUS system-of-systems deployment at the DARPA Subterranean Challenge Tunnel and Urban Circuits, along with the current limitations and the lessons learned for the benefit of the community., Comment: 50 pages, 25 figures. Accepted at Field Robotics, 2021

This paper contributes a method to design a novel navigation planner exploiting a learning-based collision prediction network. The neural network is tasked to predict the collision cost of each action sequence in a predefined motion primitives library in the robot's velocity-steering angle space, given only the current depth image and the estimated linear and angular velocities of the robot. Furthermore, we account for the uncertainty of the robot's partial state by utilizing the Unscented Transform and the uncertainty of the neural network model by using Monte Carlo dropout. The uncertainty-aware collision cost is then combined with the goal direction given by a global planner in order to determine the best action sequence to execute in a receding horizon manner. To demonstrate the method, we develop a resilient small flying robot integrating lightweight sensing and computing resources. A set of simulation and experimental studies, including a field deployment, in both cluttered and perceptually-challenging environments is conducted to evaluate the quality of the prediction network and the performance of the proposed planner., Comment: 8 pages, 7 figures. Accepted at the IEEE International Conference on Robotics and Automation (ICRA), 2022

This work investigates the effect of double intelligent reflecting surface (IRS) in improving the spectrum efficient of multi-user multiple-input multiple-output (MIMO) network operating in the millimeter wave (mmWave) band. Specifically, we aim to solve a weighted sum rate maximization problem by jointly optimizing the digital precoding at the transmitter and the analog phase shifters at the IRS, subject to the minimum achievable rate constraint. To facilitate the design of an efficient solution, we first reformulate the original problem into a tractable one by exploiting the majorization-minimization (MM) method. Then, a block coordinate descent (BCD) method is proposed to obtain a suboptimal solution, where the precoding matrices and the phase shifters are alternately optimized. Specifically, the digital precoding matrix design problem is solved by the quadratically constrained quadratic programming (QCQP), while the analog phase shift optimization is solved by the Riemannian manifold optimization (RMO). The convergence and computational complexity are analyzed. Finally, simulation results are provided to verify the performance of the proposed design, as well as the effectiveness of double-IRS in improving the spectral efficiency., Comment: This work has been accepted by IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY

This paper presents a novel strategy for autonomous teamed exploration of subterranean environments using legged and aerial robots. Tailored to the fact that subterranean settings, such as cave networks and underground mines, often involve complex, large-scale and multi-branched topologies, while wireless communication within them can be particularly challenging, this work is structured around the synergy of an onboard exploration path planner that allows for resilient long-term autonomy, and a multi-robot coordination framework. The onboard path planner is unified across legged and flying robots and enables navigation in environments with steep slopes, and diverse geometries. When a communication link is available, each robot of the team shares submaps to a centralized location where a multi-robot coordination framework identifies global frontiers of the exploration space to inform each system about where it should re-position to best continue its mission. The strategy is verified through a field deployment inside an underground mine in Switzerland using a legged and a flying robot collectively exploring for 45 min, as well as a longer simulation study with three systems., Comment: 8 pages, 5 figures. Accepted for the IEEE International Conference on Robotics and Automation, 2022. Code available at https://github.com/ntnu-arl/gbplanner_ros

This paper presents a review of the design and application of model predictive control strategies for Micro Aerial Vehicles and specifically multirotor configurations such as quadrotors. The diverse set of works in the domain is organized based on the control law being optimized over linear or nonlinear dynamics, the integration of state and input constraints, possible fault-tolerant design, if reinforcement learning methods have been utilized and if the controller refers to free-flight or other tasks such as physical interaction or load transportation. A selected set of comparison results are also presented and serve to provide insight for the selection between linear and nonlinear schemes, the tuning of the prediction horizon, the importance of disturbance observer-based offset-free tracking and the intrinsic robustness of such methods to parameter uncertainty. Furthermore, an overview of recent research trends on the combined application of modern deep reinforcement learning techniques and model predictive control for multirotor vehicles is presented. Finally, this review concludes with explicit discussion regarding selected open-source software packages that deliver off-the-shelf model predictive control functionality applicable to a wide variety of Micro Aerial Vehicle configurations.

In this paper we present an overview of the methods and systems that give rise to a flying robotic system capable of autonomous inspection, surveying, comprehensive multi-modal mapping and inventory tracking of construction sites with high degree of systematicity. The robotic system can operate assuming either no prior knowledge of the environment or by integrating a prior model of it. In the first case, autonomous exploration is provided which returns a high fidelity $3\textrm{D}$ map associated with color and thermal vision information. In the second case, the prior model of the structure can be used to provide optimized and repetitive coverage paths. The robot delivers its mapping result autonomously, while simultaneously being able to detect and localize objects of interest thus supporting inventory tracking tasks. The system has been field verified in a collection of environments and has been tested inside a construction project related to public housing., Comment: 4 pages, 8 figures

This paper presents the design concept, modeling and motion planning solution for the aerial robotic chain. This design represents a configurable robotic system of systems, consisting of multi-linked micro aerial vehicles that simultaneously presents the ability to cross narrow sections, morph its shape, ferry significant payloads, offer the potential of distributed sensing and processing, and allow system extendability. We contribute an approach to address the motion planning problem of such a connected robotic system of systems, making full use of its reconfigurable nature, to find collision free paths in a fast manner despite the increased number of degrees of freedom. The presented approach exploits a library of aerial robotic chain configurations, optimized either for cross-section size or sensor coverage, alongside a probabilistic strategy to sample random shape configurations that may be needed to facilitate continued collision-free navigation. Evaluation studies in simulation involve traversal of constrained and obstacle-laden environments, having narrow corridors and cross sections.

This paper investigates a secure wireless-powered device-to-device (D2D) communication network in the presence of multiple eavesdroppers, where a hybrid base station (BS) in a cellular network not only provides power wirelessly for the D2D transmitter to guarantee power efficiency for the D2D network, but also serves as a cooperative jammer (CJ) to interfere with the eavesdroppers. The cellular and D2D networks can belong to different service providers, which means that the D2D transmitter would need to pay for the energy service released by the hybrid BS to guarantee secure D2D communication. In order to exploit the hierarchical interaction between the BS and the D2D transmitter, we first formulate a Stackelberg game based energy trading scheme, where the quadratic energy cost model is considered. Then, a non-energy trading based Stackelberg game is investigated to study the reversed roles of the BS and the D2D users. For comparison, we also formulate and resolve the social welfare optimization problem. We derive the closed-form Stackelberg equilibriums of the formulated games and the optimal solutions for the social welfare optimization problem. Simulation results are provided to validate our proposed schemes to highlight the importance of energy trading interaction between cellular and D2D networks.