Search

International audience; Over the past decade, the Differential Dynamic Programming (DDP) method has gained in maturity and popularity within the robotics community. Several recent contributions have led to the integration of constraints within the original DDP formulation, hence enlarging its domain of application while making it a strong and easy-to-implement competitor against alternative methods of the state of the art such as collocation or multiple-shooting approaches. Yet, and similarly to its competitors, DDP remains unable to cope with high-dimensional dynamics within a receding horizon fashion, such as in the case of online generation of athletic motion on humanoid robots. In this paper, we propose to make a step towards this objective by reformulating classical DDP as an implicit optimal control problem, allowing the use of more advanced integration schemes such as implicit or variational integrators. To that end, we introduce a primal-dual proximal Lagrangian approach capable of handling dynamical and path constraints in a unified manner, while taking advantage of the time sparsity inherent to optimal control problems. We show that this reformulation enables us to relax the dynamics along the optimization process by solving it inexactly: far from the optimality conditions, the dynamics are only partially fulfilled, but continuously enforced as the solver gets closer to the local optimal solution. This inexactness enables our approach to robustly handle large time steps (100 ms or more), unlike other DDP solvers of the state of the art, as experimentally validated through different robotic scenarii.

In this paper, we propose a method that shows how phase portraits rendered by a controller can inform the development of a physical adaptation at a single degree of freedom (DoF) for a given control task. This approach has the advantage of having physical adaptations sharing the responsibility of control to accomplish a task. We use an inverted pendulum which is reminiscent of the trunk of a biped walker to conduct numerical simulations and hardware experiments to show how our method can innovate a physical adaptation at the pivot joint to reduce the control effort. Our method discovered that a torsional spring at the pivot joint would lead to a lower input effort by the regulator type feedback controller. The method can tune the spring to minimize the total cost of control up to about 32.81%. This physical adaptation framework allows multiple degrees of freedom robotic system to suggest local physical adaptations to accomplish a given control objective.

This work presents a machine vision system for the localization of strawberries and environment perception in a strawberry-harvesting robot for use in table-top strawberry production. A deep convolutional neural network for segmentation is utilized to detect the strawberries. Segmented strawberries are localized through coordinate transformation, density base point clustering and the proposed location approximation method. To avoid collisions between the gripper and fixed obstacles, the safe manipulation region is limited to the space in front of the table and underneath the strap. Therefore, a safe region classification algorithm, based on Hough Transform algorithm, is proposed to segment the strap masks into a belt region in order to identify the pickable strawberries located underneath the strap. Similarly, a safe region classification algorithm is proposed for the table, to calculate its points in 3D and fit the points onto a 3D plane based on the 3D point cloud, so that pickable strawberries in front of the table can be identified. Experimental tests showed that the algorithm could accurately classify ripe and unripe strawberries and could identify whether the strawberries are within the safe region for harvesting. Furthermore, harvester robot's optimized localization method could accurately locate the strawberry targets with a picking accuracy rate of 74.1% in modified situations.

Traditionally, the assessment of upper limb (UL) motor function in neurorehabilitation is carried out by clinicians using standard clinical tests for objective evaluation, but which could be influenced by the clinician's subjectivity or expertise. The automation of such traditional outcome measures (tests) is an interesting and emerging field in neurorehabilitation. In this paper, a systematic review of systems focused on automation of traditional tests for assessment of UL motor function used in neurological rehabilitation is presented. A systematic search and review of related articles in the literature were conducted. The chosen works were analyzed according to the automation level, the data acquisition systems, the outcome generation method, and the focus of assessment. Finally, a series of technical requirements, guidelines, and challenges that must be considered when designing and implementing fully-automated systems for upper extremity functional assessment are summarized. This paper advocates the use of automated assessment systems (AAS) to build a rehabilitation framework that is more autonomous and objective. This work was supported in part by the Spanish Ministry of Economy and Competitiveness via the ROBOHEALTH (DPI2013-47944-C4-1-R) and ROBOESPAS (DPI2017-87562-C2-1-R) Projects, and in part by the RoboCity2030-III-CM project (S2013/MIT-2748) which is funded by the Programas de Actividades I+D Comunidad de Madrid and cofunded by the Structural Funds of the EU.

Fiducial markers are commonly used to localize robots. Most existing systems use standard cameras to detect simple patterns on flat tags. Since depth is not directly sensed, the pose must be inferred from the tag geometry. These systems are low-cost and easy to implement on robotic systems, but their performances suffer in low-light conditions and on computationally constrained processors. We propose using 3D light detection and ranging (LiDAR) scanners to mitigate these issues. The reflectivity measurement provided by most LiDAR sensors provides a simple way to discern geometric differences on surfaces. We utilize this fact to create a custom “beacon” with reflective fiducials. Next, we design a high-performance segmentation and localization algorithm to find the 2D pose of a mobile robot. Our experiments proved that our system achieves an average euclidean error of less than 0.063 m at ranges of over 10 m while maintaining a runtime of under 3 ms on a basic single board computer. Additionally, our system is highly occlusion resistant. These results are confirmed with multiple field tests of the system.

This research work mainly focuses on the study of the modeling and control of microrobotic systems in a biomedical context. So far, the use of magnetic actuation has been regarded as the most convenient approach for such achievements. Besides, the cardiovascular system allows to reach most parts of the human body and is then chosen as the main navigation route. This original topic is a rapidly expanding field whose ambition is to modernize current therapies by trying to improve therapeutic targeting while improving patient comfort. To achieve this goal, a good understanding of how microrobots evolve in the human body is an important step. The theoretical foundations and the physical laws that make it possible to describe the various phenomena which act on magnetic microrobots in vascular-like environments have thus been deeply studied. Methodologies for dealing with multiphysics approaches combining different sources of hypotheses and uncertainties have been developed. Great care has been taken in their validations by experimentation when possible, otherwise by numerical analysis. This helps to better understand the dominant dynamics, as well as the predominant parameters in the description of magnetic microrobots in a vascular-like environment. This makes it possible to efficiently characterize and predict their behaviors in a viscous flow and their responses to magnetic fields. On this basis, advanced navigation strategies have been developed. The navigation process can be divided into two stages. First, safe and efficient navigation paths are planned (off-line) based on the fast marching method (FMM). With the proposed navigation planning framework, different constraints and objectives can then be taken into account to obtain a truly feasible reference path. Second, control schemes that drive the magnetic microrobots along the planned reference path to the targeted location are synthesized. To do so, predictive and optimal control laws have been implemented. All the proposed models and navigation strategies have been evaluated through various experiments under different conditions with the platforms developed at the PRISME Laboratory., Ce travail traite principalement l'étude de la modélisation et du contrôle de systèmes microrobotiques dans un contexte biomédical. Jusqu'à présent, l'utilisation de l'actionnement magnétique est considérée comme l'approche la plus prometteuse pour de telles réalisations. Par ailleurs, le système cardiovasculaire permet d'atteindre la plupart des parties du corps humain et est alors choisi comme voie de navigation principale. Cette thématique originale est un domaine en pleine expansion dont l'ambition est de moderniser les thérapies actuelles en essayant d'améliorer le ciblage thérapeutique tout en améliorant le confort des patients. Pour atteindre cet objectif, une bonne compréhension du comportement des microrobots dans le corps humain est une étape importante. Les fondements théoriques et les lois de la physique permettant de décrire les différents phénomènes qui agissent sur les microrobots magnétiques dans des environnements de type vasculaire ont ainsi été étudiés en détails. Des méthodologies pour traiter des approches multiphysiques combinant différentes sources d'hypothèses et d'incertitudes ont été développées. Un grand soin a été apporté à leurs validations par l'expérimentation lorsque cela était possible, le cas échéant par des analyses numériques. Cela permet de mieux comprendre la dynamique dominante, ainsi que les paramètres prédominants dans la description des microrobots magnétiques. Cela permet de caractériser et de prédire efficacement leurs comportements dans un écoulement visqueux et leurs réponses aux champs magnétiques. Sur cette base, des stratégies de navigation avancées ont été développées. Le processus de navigation peut être divisé en deux étapes. Premièrement, des chemins de navigation sûrs et efficaces sont planifiés (hors ligne) sur la base de la méthode du _[fast marching]{lang=en .en}_ (FMM). Dans le cadre de la méthode de planification de la navigation proposée, différentes contraintes et objectifs peuvent être pris en compte pour obtenir un chemin de référence véritablement réalisable. Deuxièmement, des stratégies de commande guidant les microrobots magnétiques le long du chemin de référence jusqu'à l'emplacement ciblé sont synthétisées. Pour ce faire, des lois de commande prédictives et optimales ont été mises en œuvre. Tous les modèles et stratégies de navigation proposés ont été évalués à travers diverses expérimentations dans différentes conditions sur les plateformes développées au Laboratoire PRISME.

Mobile manipulators are changing the way companies and industries complete their work. Untrained end users risk facing unfunctional and nonuser- friendly Graphical User Interfaces. Recently, there has been shortages of people and talent in the heathcare industry where these applications would benefit in being used to accomplish easy and low level tasks. All these reasons contribute to the need of finding functional robot-user ways of communicating that allow the expansion of mobile manipulation applications. This thesis addresses the problem of finding an intuitive way to deploy a mobile manipulator in a laboratory environment. This thesis has analyzed whether it is possible to permit the user to work with a manipulator efficiently and without too much effort via a functional graphical user interface. Creating a modular interface based on user needs is the innovation value of this work. It allows the expansion of mobile manipulator applications that increases the number of possible users. To accomplish this purpose a Graphical User Interface application is proposed using an explanatory research strategy. First, user data was acquired using an ad hoc research survey and mixed with literature implementations to create the right application design. Then, an iterative implementation based on code-creation and tests was used to design a valuable solution. Finally, the results from an observational user study with non-roboticist programmers are presented. The results were validated with the help of 10 potential end users and a validation matrix. This demonstrated how the system is both functional and user-friendly for novices, but also expressive for experts. Mobilmanipulatorer förändrar sättet som företag och industrier utför sitt arbete. Otränade slutanvändare och särskilt de utan programmeringskunskap kommer att bemötas av icke-funktionella och användarovänliga grafiska användargränssnitt. Den senaste tiden har det varit brist på specialiserad personal inom hälsovårdsindustrin som har resulterat i ett beroende på dessa applikationer för att genomföra enkla uppgifter samt uppgifter på låg nivå. Alla dessa faktorer bidrar till det ökande behovet att hitta ett funktionellt sätt att kommunicera mellan robot och slutanvändare vilket tillåter expansionen av mobilmanipulatorapplikationer. Arbetet som beskrivs i denna avhandling adresserar problemet att finna ett intuitivt sätt att använda en mobilmanipulator i ett laboratoriemijö. Möjligheten att tillåta användaren att på ett enkelt och effektivt sätt arbeta med en manipulator via ett funtionellt grafiskt användargränssnitt analyseras. Innovationsvärdet och detta examensarbetes bidrag till nuvarande kunskap betraktar möjligheten att skapa ett modulärt gränssnitt baserat på användares behov. Detta möjliggör expansionen av mobilmanipulatörers applikation vilket ökar antalet möjliga användare. En förklarande forskningsstrategi används för att föreslå en grafisk användargränssnittsapplikation för att uppnå detta mål. Först användes data från ad hoc-undersökningar blandat med litteraturimplementeringar för att skapa den rätta applikationsdesignen. En iterativ implementering baserad på kodskapande samt tester användes sedan för att designa en värdefull lösning redo att testas. Slutligen presenteras resultat från en användarobservationsstudie med icke-robotikprogrammerare. De insamlade resultaten som samlades in under valideringsstadiet tack vare en grupp bestående av tio potentiella slutanvändare har analyserats genom användandet av en valideringsmatris som är baserad på tre parametrar. Detta demonstrerade hur systemet är både funktionellt och användarvänligt för nybörjare men också expressivt för experter.

Food Logistics 4.0 is a term derived from Industry 4.0 focusing on all the aspects of food logistics management based on cyber-physical systems. It states that real-time information and the interconnectivity of things, supplemented with novel technologies will revolutionise and improve the way food logistics is carried out. It has tremendous potential in terms of bringing transparency, swift delivery of food at reduced cost, flexibility, and capability to deliver the right quality product at the right place and at the right time. This paper discusses the vital technologies within Food Logistics 4.0 and the opportunities and challenges in this regard. It focuses primarily on food logistics, including resource planning, warehouse management, transportation management, predictive maintenance, and data security. Internet of Things, Blockchain, Robotics and Automation and artificial intelligence are some of the technologies discussed.

The design and test of a robot-compatible, radiation detection instrument providing simultaneous $\gamma $ -ray imaging and $\gamma $ -ray spectroscopy is described. The sensing system comprises a cerium bromide inorganic scintillation detector and a cylindrical, lead slot collimator that is configured with a robot-compatible, on-board data acquisition system. The mount for the sensor is a lightweight, bespoke 3-axis gimbal actuated by servos for pan, tilt and rotation of the collimator for imaging capability. This paper discusses the integration of this relatively low-cost radiation detection apparatus with a commercially available, Robot-Operating-System-controlled robotic platform (a Clearpath Robotics™ Jackal). The detection system is compliant with the power and mass payload constraints of the robot. Its performance has been evaluated by means of two practical examples: a) measurements in a laboratory environment to assess the ability of the system to resolve two caesium-137 point sources, and b) deployment at the Jožef Stefan Institute TRIGA Mark II research reactor to assess the ability of the system to characterise the $\gamma $ -ray emission at 1 kW from a horizontal tangential beam port in the reactor hall and from the reactor sample pool above the core.

Monocular vision systems allow compact and resource-constrained vehicles to perform online state estimation through Structure from Motion approaches. However, it is needed to integrate additional information to obtain a metric scale for the estimated trajectory. We propose a minimalistic approach based on the sensor fusion between a monocular camera and a 1D LiDAR (LIght Detection And Ranging) rangefinder. A classical visual front-end, in charge of tracking the camera pose with respect to a map of landmarks, is paired with an optimization back-end where depth measurements constrain the distance between cameras and 3D points. The fusion methodology is based on a keyframe-wise search of correspondences between visual landmarks and projections of altimeter points. The differences between triangulated landmark depths and associated altimeter measurements are minimized using the iSAM2 incremental smoother. We test our algorithm in a variety of datasets comprising an outdoor scenario with accurate D-GPS ground truth evaluating both tracking accuracy and computational effort. Results show that our algorithm provides comparable performances to scale-aware RGB-D vision systems and obtains relative trajectory errors close to 0.5%. Furthermore, we propose and validate an extrinsic calibration pipeline to refer range measurements in the camera reference system which differs from common LiDAR-camera calibration algorithms due to the 5 Degrees of Freedom (DoF) nature of the single-point rangefinder and camera system.

Recent research and developments in cloud robotics (CR) require appropriate knowledge representation to ensure interoperable data, information, and knowledge sharing within cloud infrastructures. As an important branch of the Internet of Things (IoT), these demands to advance it forward motivates academic and industrial sectors to invest on it. The IEEE ’Ontologies for Robotics and Automation’ Working Group (ORA WG) has been developing standard ontologies for different robotic domains, including industrial and autonomous robots. The use of such robotic standards has the potential to benefit the Cloud Robotic Community (CRC) as well, supporting the provision of ubiquitous intelligent services by the CR-based systems. This paper explores this potential by developing an ontological approach for effective information sharing in cloud robotics scenarios. It presents an extension to the existing ontological standards to cater for the CR domain. The use of the new ontological elements is illustrated through its use in a couple of CR case studies. To the best of our knowledge, this is the first work ever that implements an ontology comprising concepts and axioms applicable to the CR domain., This work was partly supported by Innovate UK; CNPq and CAPES in Brazil; FRGS/1/2015/TK08/MUSM/02/1 in Malaysia.

The COVID-19 pandemic unraveled the weak points in the global supply chain for goods. Specifically, people all over the world, including those in the most advanced nations have had to go without medical supplies and personal protective equipment. Scarcity of essentials increases anxiety and uncertainty exacerbating unproductive behaviors like hoarding and price gouging. Left to market forces, such unfair practices are likely to aggravate hardships and increase the loss of lives. Thus, there is a critical need to ensure safe distribution of food and essential supplies to all citizens to sustain them through challenging times. To this end, we propose a simple, affordable and contact-less robotic system for preparing and dispensing food and survival-kits at community scale. The system has provisions to prevent hoarding and price gouging. Design, simulation, and, validation of the system has been completed to ensure readiness for real world implementation. This project is part of an open-source program and detailed designs are available upon request to entities interested in using it to serve their communities.

The 3-D reconstruction involves the construction of a 3-D model from a set of images. The 3-D reconstruction has varied uses that include 3-D printing, the generation of 3-D models that can be shared through social media, and more. The 3-D reconstruction involves complex computations in mobile phones that must determine the pose estimation. The pose estimation involves the process of transforming a 2-D object into 3-D space. Once the pose estimation is done, then the mesh generation is performed using the graphics processing unit. This helps render the 3-D object. The competitive advantages of using hardware processors are to accelerate the intensive computation using graphics processors and digital signal processors. The stated problem that this technical paper addresses is the need for a reliable automated test for the 3-D reconstruction feature. The solution to this problem involved the design and development of an automated test system using a programmable robotic arm and rotor for precisely testing the quality of 3-D reconstruction features. The 3-D reconstruction testing involves using a robotic arm lab to accurately test the algorithmic integrity and end-to-end validation of the generated 3-D models. The robotic arm can move the hardware at different panning speeds, specific angles, fixed distances from the object, and more. The ability to reproduce the scanning at a fixed distance and the same panning speed helps to generate test results that can be benchmarked by different software builds. The 3-D reconstruction also requires a depth sensor to be mounted onto the device under examination. We use this robotic arm lab for functional, high performance, and stable validation of the 3-D reconstruction feature. This paper addresses the computer vision use case testing for 3-D reconstruction features and how we have used the robotic arm lab for automating these use cases.

Recently, stereovision has been used in robotics for obtaining information for real-time mapping. Stereovision makes use of two cameras to solve the problems of a lack of discriminative image features and repetitive patterns and to extend the field of view. Stereovision algorithms and techniques are working to create a map to know its surrounding environment. Despite advances in the field of autonomous robotics, several challenges still exist. The majority of these challenges exist owing to the dynamic nature of the navigational environment and the lack of complete knowledge of new, unknown, and complex environments. In this paper, we create a novel, intelligent, and dynamic mapping system for autonomous navigation using stereovision. The proposed system is called intelligent multi-sensor multi-baseline mapping system (I3MS). The main contribution of this paper is the dynamic tuning of the stereovision parameters to address the dynamics of the environment (various light conditions, movement obstacles, and various times of day). The I3MS works in unstructured, unknown, dynamic, and indoor environments. It is based on an in-depth study of the effects of light and distance from obstacles on the accuracy of the 3-D map. It uses a fuzzy-logic system to tune the stereovision parameters to handle dynamic environments. The obtained experimental results illustrate the robustness and accuracy of the proposed system.

Touch-to-track is a software feature that is used to track objects in embedded systems, such as mobile phones. The research question that is discussed in the paper is as follows: how can one design an automated test method for accurately testing object tracking algorithms? The challenge for the verification team was to design a method of test automation for testing this object tracking feature. The solution to the problem was to develop test automation algorithms using a robotic arm to accurately test this software feature. The robotic arm is used for executing test cases for tracking single or multiple objects in motion. To use the touch-to-track feature, the user selects an area in the camera preview by drawing a bounding box. Hough transformation and color segmentation are applied to accurately detect the bounding box drawn on every frame in which the object is detected. The area inside the bounding box is considered as the template. Template matching algorithms based on normalized cross-correlation, phase correlation, and speeded up robust features (SURF)-based feature extraction and matching are then applied to compare the template and original images. The tracking efficiency is calculated by dividing the total number of frames that contain the object being tracked by the total number of frames being tested. The tracking efficiencies achieved with the Hough-based bounding box detection algorithm followed by template matching algorithms based on normalized cross-correlation, phase correlation, and SURF-based feature extraction and matching are compared. These tracking efficiencies are calculated by comparing the ground truth to the tracking results for each frame. The tracking efficiency calculated for three objects is 87.7% with no colored background. The proposed object tracking solution with hardware acceleration is found to perform better than the third-party solution with respect to the latency in re-establishing tracking and the hand jitter scenario.

Despite significant research progress on small-scale aerial–aquatic robots, most existing prototypes are still constrained by short operation times and limited performance in different fluids. The main challenge is to design a vehicle that satisfies the partially conflicting design requirements for aerial and aquatic operations. In this letter we present a new class of aerial–aquatic robot, the sailing micro air vehicle, “SailMAV.” Thanks to a three-part folding wing design, the SailMAV is capable of both flying and sailing. The robot design permits long and targeted missions at the water interface by leveraging the wind as movement vector. It simultaneously offers the flexibility of flight for rapidly reaching a designated area, overcoming obstacles, and moving from one body of water to another, which can be very useful for water sampling in areas with many obstacles. With a total wingspan of 0.96 m, the SailMAV employs the same wing and actuation surfaces for sailing as for flying. It is capable of water surface locomotion as well as takeoff and flight at a cruising speed of 10.8 m $\cdot$ s $^{-1}$ . The main contributions of this letter are new solutions to the challenges of combined aerial and aquatic locomotions, the design of a novel hybrid concept, the development of the required control laws, and the demonstration of the vehicle successfully sailing and taking off from the water. This letter can inform the design of hybrid vehicles that adapt their morphology to move effectively.

Within the next decades, robots will need to be able to execute a large variety of tasks autonomously in a large variety of environments. To relax the resulting programming effort, a knowledge-enabled approach to robot programming can be adopted to organize information in re-usable knowledge pieces. However, for the ease of reuse, there needs to be an agreement on the meaning of terms. A common approach is to represent these terms using ontology languages that conceptualize the respective domain. In this work, we will review projects that use ontologies to support robot autonomy. We will systematically search for projects that fulfill a set of inclusion criteria and compare them with each other with respect to the scope of their ontology, what types of cognitive capabilities are supported by the use of ontologies, and which is their application domain., This work was partially funded by the following agencies: Deutsche Forschungsgemeinschaft (DFG) through the Collaborative Research Center EASE (1320); Regional Catalan Agency ACCIO through the RIS3CAT 2016 project SIMBIOTS (COMRDI16-1-0017); Spanish State ´ Research Agency through the Mar´ıa de Maeztu Seal of Excellence to IRI (MDM-2016-0656); Spanish Government through the project DPI2016-80077-R and grant 2017; FCT, through IDMEC, under LAETA, project UID/EMS/50022/2019; Project 0043- EUROAGE-4-E (POCTEP Programa Interreg V-A Spain-Portugal). J. Olszewska was partially supported by Innovate UK.