Search

Technological advancements have facilitated the development of sophisticated vision systems, integrating optical sensors with artificial vision and machine learning techniques to create applications in different fields of robotics. One such field is Search and Rescue (SAR) robotics, which has historically played a significant role in assisting brigades following post-disaster events, particularly in exploration phases and, crucially, in victim identification. The importance of employing these systems in victim identification lies in their functionality under challenging conditions, enabling the capture of information across different light spectrum ranges (RGB, Thermal, Multispectral). This article proposes an innovative comparative analysis that scrutinizes the advantages and limitations of three sensor types in victim detection. It explores contemporary developments in the state-of-the-art and proposes new metrics addressing critical aspects, such as functionality in specific scenarios and the analysis of environmental disturbances. For the indoor and outdoor testing phase, a quadrupedal robot has been equipped with these cameras. The primary findings highlight the individual contributions of each sensor, particularly emphasizing the efficacy of the infrared spectrum for the thermal camera and the Near Infrared and Red Edge bands for the multispectral camera. Ultimately, following system evaluations, detection precisions exceeding 92% and 86%, respectively, were achieved.

The technological advancements in sensory systems and robotics over the past decade have facilitated the innovation of centralized systems for optimizing resource utilization and monitoring efficiency in inspection applications. This paper presents a novel system designed for gas concentration sensing in environments by implementing a modular artificial nose (emulating the inhalation and exhalation process) equipped with a strategically designed air capture centralization system based on computational fluid dynamics analysis (CFD). The system incorporates three gas identification sensors distributed within the artificial nose, and their information is processed in real-time through embedded systems. The artificial nose is hardware-software integrated with a quadruped robot capable of traversing the environment to collect samples, maximizing coverage area through its mobility and locomotion capabilities. This integration provides a comprehensive perspective on gas distribution in a specific area, enabling the efficient detection of substances in the surrounding environment. The robotic platform employs a graphical interface for real-time gas concentration data map visualization. System integration is achieved using the Robot Operating System (ROS), leveraging its modularity and flexibility advantages. This innovative robotic approach offers a promising solution for enhanced environmental inspection and monitoring applications.

Early detection of victims in post-disaster environments is an essential task for rescue teams, and effective victim identification will save the greatest number of lives. One of the most significant challenges to carry out these tasks lies mainly in the harsh conditions typical of post-disaster environments such as low light, victims trapped both in rubble and by some materials that make their location difficult (blankets, wood, glass, Etc.) by simple visual inspection. To face this problem, this Chapter describes a robust method for identifying victims and their location within a heat map of the analyzed environment, using specific equipment (Thermal Camera Optris PI640, Microsoft Kinect, and Robot Dr. Jaguar). For this task development, a novel method developed in ROS is proposed based on the processing of thermal and depth images, in such a way that the first ones are used to detect victims through a Convolutional Neural Network and to generate Thermal Local Point Clouds of the environments in combination with depth images. Iterative Closest Point algorithms were used for a subsequent complete thermal map concatenation of the 3D environment using Point Clouds (Thermal Global Point Clouds). The tests to validate this method were carried out indoors (CAR Arena at Centre for Automation and Robotics) and outdoors (Robotics and Cybernetic Laboratory), recreating environments with post-disaster conditions. The main results show an average of 89% in concatenation efficiency to generate Thermal Global Point Clouds. For different scenarios. On the other hand, the detection of victims was carried out with an efficiency greater than 90% for the different proposed scenarios.

Post-disaster environments resulting from catastrophic events, leave sequels such as victims trapped in debris, which are difficult to detect by rescuers in a first inspection. Technological advances in electronics and perception have allowed the development of versatile and powerful optical sensors capable of capturing light in spectrums that humans cannot. new deep learning techniques, such as convolutional neural networks (CNNs), has allowed the generation of network models capable of autonomously detecting specific image patterns according to previous training. This work introduces an autonomous victim detection system to be deployed by using search and rescue robots. The proposed system defines new indexes based on combining the multispectral bands (Blue, Green, Red, Nir, Red Edge) to obtain new multispectral images where relevant characteristics of victims and the environment are highlighted. CNNs have been used as a second phase for automatically detecting victims in these new multispectral images. A qualitative and quantitative analysis of new indexes proposed by the authors has been carried out to evaluate their efficiency in contrast to the state-of-the-art ones. A data set has been generated to train different CNN models based on the best obtained index to analyze their effectiveness in detecting victims. The results show an efficiency of 92% in automatically detecting victims when applying the best multispectral index to new data. This method has also been contrasted with others based on thermal and RGB imagery to detect victims, where it has been proven that it generates better results in situations of outdoor environments and different weather conditions.

Technological breakthroughs in recent years have led to a revolution in fields such as Machine Vision and Search and Rescue Robotics (SAR), thanks to the application and development of new and improved neural networks to vision models together with modern optical sensors that incorporate thermal cameras, capable of capturing data in post-disaster environments (PDE) with rustic conditions (low luminosity, suspended particles, obstructive materials). Due to the high risk posed by PDE because of the potential collapse of structures, electrical hazards, gas leakage, etc., primary intervention tasks such as victim identification are carried out by robotic teams, provided with specific sensors such as thermal, RGB cameras, and laser. The application of Convolutional Neural Networks (CNN) to computer vision is a breakthrough for detection algorithms. Conventional methods for victim identification in these environments use RGB image processing or trained dogs, but detection with RGB images is inefficient in the absence of light or presence of debris; on the other hand, developments with thermal images are limited to the field of surveillance. This paper's main contribution focuses on implementing a novel automatic method based on thermal image processing and CNN for victim identification in PDE, using a Robotic System that uses a quadruped robot for data capture and transmission to the central station. The robot's automatic data processing and control have been carried out through Robot Operating System (ROS). Several tests have been carried out in different environments to validate the proposed method, recreating PDE with varying conditions of light, from which the datasets have been generated for the training of three neural network models (Fast R-CNN, SSD, and YOLO). The method's efficiency has been tested against another method based on CNN and RGB images for the same task showing greater effectiveness in PDE main results show that the proposed method has an efficiency

Recent developments in bioinspired technologies combined with the advance of intelligent and soft materials have allowed soft robots to replicate the behavior of different animal species. These devices can perform complicated tasks such as reaching or adapting in constrained and unstructured environments. This article proposes a methodology to develop a soft robot called “JellyRobcib” inspired in morphology and behavior by jellyfish, using shape-memory alloy springs as actuators (as bio-muscles). Such actuators can move the jellyfish both vertically and laterally by applying closed-loop fuzzy and visual controls. Additionally, Computer-Assisted Designs and Computational Fluid Dynamics simulations have been carried out to validate the soft robot model. The results show that the robot movements are very close to the morphological behavior of a real jellyfish regarding the curves of displacements, speeds and accelerations, after performing several experiments for autonomous movement: vertical ascent, lateral movements and trajectory tracking, obtaining an accuracy of ±1479 cm and repeatability of 0.944 for lateral movements for fuzzy visual control. Furthermore, thermal measurements were taken throughout a given path, allowing the generation of temperature gradients within the underwater environment for monitoring purposes.

Especificaciones de diseño y funcionamiento de una plataforma robótica para fertilizar planta a planta un cultivo ecológico

Mechanization in organic strip-crops is difficult due to plant layout on the ground. In this work, a prototype of automated robot for providing organic fertiliser at single plant scale is described.

This paper analyzes three path planning techniques to solve a ball-maze system with a two-degree-of-freedom platform. The system’s objective focuses on the ball traveling a path from an initial point to a final point (defined by the user) in the maze. The RRT algorithms (Rapidly Exploring Random Trees), PRM (Probabilistic Roadmap), and Voronoi diagrams were implemented using the A* search algorithm. The system architecture consists of four subsystems called mechanical, vision, planning, and control. The main contribution of this work is the evaluation of the algorithms on a physical system and a complete results analysis (graphical and analytical). The experimental tests were performed based on analyzing four different maze configurations, the run time, and path length metrics. In this context, 20 algorithm executions were developed for each configuration, then the meantime and mean length and their 95% confidence intervals were determined. The main results show that the RRT algorithm presents a more significant variation in its data, the longest path length, and the best performance in terms of run time. Moreover, the PRM algorithm generates the path with the shortest length but has the worst performance concerning run time. Finally, the Voronoi diagrams’ technique takes less time to execute, has less variation in its data, and presents the smoothest and equidistant path between the maze walls. © 2022, Universidad de Tarapaca.

The development of new sensory and robotic technologies in recent years and the increase in the consumption of organic vegetables have allowed the generation of specific applications around precision agriculture seeking to satisfy market demand. This article analyzes the use and advantages of specific optical sensory systems for data acquisition and processing in precision agriculture for Robotic Fertilization process. The SUREVEG project evaluates the benefits of growing vegetables in rows, using different technological tools like sensors, embedded systems, and robots, for this purpose. A robotic platform has been developed consisting of Laser Sick AG LMS100 x 3, Multispectral, RGB sensors, and a robotic arm equipped with a fertilization system. Tests have been developed with the robotic platform in cabbage and red cabbage crops, information captured with the different sensors, allowed to reconstruct rows crops and extract information for fertilization with the robotic arm. The main advantages of each sensory have been analyzed with an quantitative comparison, based on information provided by each one; such as Normalized Difference Vegetation Index index, RGB Histograms, Point Cloud Clusters). Robot Operating System processes this information to generate trajectory planning with the robotic arm and apply the individual treatment in plants. Main results show that the vegetable characterization has been carried out with an efficiency of 93.1 using Point Cloud processing, while the vegetable detection has obtained an error of 4.6 through RGB images.

The growing demand for organically produced vegetables requires the adoption of new cropping systems such as strip-cropping. To counteract the additional labour mixed cropping entails, automation and robotics play a key role. This research focuses on the development of a proof-of-concept platform that combines optical sensors and an actuation system for targeted precision fertilization that encircles selected plants rather than a local field area. Two sensor types are used for the detection of a fertilisation need: a multispectral camera and light detection and ranging (LiDAR) devices in order to acquire information on plant health status and three-dimensional characterisation. Specific algorithms were developed to more accurately detect a change in fertilization need. An analysis of their results yields a prescription map for automatic fertilisation through a robotic arm. The relative location of the platform within the prescription map is essential for the correct application of fertilizers, and is acquired through live comparison of a LiDAR pushbroom with the known 3D world model. The geometry of each single plant is taken into account for the application of the sprayed fertiliser. This resulted in a reliable method for the detection of delayed growth and prototype localization within a changing natural environment without relying on external markers.

In precision agriculture (PA), vegetation indices (VI) are commonly used to evaluate the health of crops with the use of multi-spectral cameras. These are often mounted on drones and used at high altitudes, where the translation of focal points and subsequent changes in perspective do not pose any difficulties. In proximal sensing, however, these translations and distortions pose a significant challenge on data processing. In this work, a Parrot Sequoia camera was mounted at a fixed height of 1.2 m and used at 3 sec and 1.5 sec intervals on an organic strip-cropping field beneath. Reference imaging revealed that the multi-spectral lenses suffer from significant barrel distortion of 30%, while the higher resolution RGB lens has a barely distinguishable 1% pincushion distortion. The subsequent field images were stitched together using an open-source panorama software to automatically detect and correct distortions. The resulting mosaics were then shifted to correct the relative position of the separate lenses, allowing for VI calculation with mm accuracy. This method allows for analysing single plants for inter-crop and even intra-crop variation, allowing the automation of tasks in an organic multicrop environment.

Advances in robotic fertilisation within the SUREVEG project

The use of robotic systems in organic farming has taken on a leading role in recent years; the Sureveg CORE Organic Cofund ERA-Net project seeks to evaluate the benefits of strip-cropping to produce organic vegetables. This includes, among other objectives, the development of a robotic tool that facilitates the automation of the fertilisation process, allowing the individual treatment (at the plant level). In organic production, the slower nutrient release of the used fertilisers poses additional difficulties, as a tardy detection of deficiencies can no longer be corrected. To improve the detection, as well as counter the additional labour stemming from the strip-cropping configuration, an integrated robotic tool is proposed to detect individual crop deficiencies and react on a single-crop basis. For the development of this proof-of-concept, one of the main objectives of this work is implementing a robust localisation method within the vegetative environment based on point clouds, through the generation of general point cloud maps (G-PC) and local point cloud maps (L-PC) of a crop row. The plants’ geometric characteristics were extracted from the G-PC as a framework in which the robot’s positioning is defined. Through the processing of real-time lidar data, the L-PC is then defined and compared to the predefined reference system previously deduced. Both subsystems are integrated with ROS (Robot Operating System), alongside motion planning, and an inverse kinematics CCD (Cyclic Coordinate Descent) solver, among others. Tests were performed using a simulated environment of the crop row developed in Gazebo, followed by actual measurements in a strip-cropping field. During real-time data-acquisition, the localisation error is reduced from 13 mm to 11 mm within the first 120 cm of measurement. The encountered real-time geometric characteristics were found to coincide with those in the G-PC to an extend of 98.6%

In this collection of videos the robotic prototype built for single plant fertilisation of strip-cropped organic field is shown in action working on real plants in the Experimental Field of UPM, with all the subsystems explained. This corressponds to deliverable D4.4 (field demo of the robotic prototype) in the framework of the SUREVEG project (Core Organic Cofund)