Search

Supplemental material, sj-docx-1-jdr-10.1177_00220345221117745 for A Multiclass Classification Model for Tooth Removal Procedures by W.M. de Graaf, T.C.T. van Riet, J. de Lange and J. Kober in Journal of Dental Research

Supplemental material, sj-docx-1-jdr-10.1177_00220345221117745 for A Multiclass Classification Model for Tooth Removal Procedures by W.M. de Graaf, T.C.T. van Riet, J. de Lange and J. Kober in Journal of Dental Research

Existing Deep Learning (DL) frameworks typically do not provide ready-to-use solutions for robotics, where very specific learning, reasoning, and embodiment problems exist. Their relatively steep learning curve and the different methodologies employed by DL compared to traditional approaches, along with the high complexity of DL models, which often leads to the need of employing specialized hardware accelerators, further increase the effort and cost needed to employ DL models in robotics. Also, most of the existing DL methods follow a static inference paradigm, as inherited by the traditional computer vision pipelines, ignoring active perception, which can be employed to actively interact with the environment in order to increase perception accuracy. In this paper, we present the Open Deep Learning Toolkit for Robotics (OpenDR). OpenDR aims at developing an open, non-proprietary, efficient, and modular toolkit that can be easily used by robotics companies and research institutions to efficiently develop and deploy AI and cognition technologies to robotics applications, providing a solid step towards addressing the aforementioned challenges. We also detail the design choices, along with an abstract interface that was created to overcome these challenges. This interface can describe various robotic tasks, spanning beyond traditional DL cognition and inference, as known by existing frameworks, incorporating openness, homogeneity and robotics-oriented perception e.g., through active perception, as its core design principles. acceptedVersion

This research was founded by the Czech Science Foundation (grant number GA 19-142375) and institutional support RVO: 61388998.

Objectives: To provide dental practitioners and researchers with a comprehensive and transparent evidence-based overview of physical robot initiatives in all fields of dentistry. Data: Articles published since 1985 concerning primary data on physical robot technology in dentistry were selected. Characteristics of the papers were extracted such as the respective field of dentistry, year of publication as well as a description of its usage. Sources: Bibliographic databases PubMed, Embase, and Scopus were searched. A hand search through reference lists of all included articles was performed. Study selection: The search timeline was between January 1985 and October 2020. All types of scientific literature in all languages were included concerning fields of dentistry ranging from student training to implantology. Robot technology solely for the purpose of research and maxillofacial surgery were excluded. In total, 94 articles were included in this systematic review. Conclusions: This study provides a systematic overview of initiatives using robot technology in dentistry since its very beginning. While there were many interesting robot initiatives reported, the overall quality of the literature, in terms of clinical validation, is low. Scientific evidence regarding the benefits, results and cost-efficiency of commercially available robotic solutions in dentistry is lacking. The rise in availability of open source control systems, compliant robot systems and the design of dentistry-specific robot technology might facilitate the process of technological development in the near future. The authors are confident that robotics will provide useful solutions in the future but, strongly, encourage an evidence-based approach when adapting to new (robot) technology.

Objectives: To provide dental practitioners and researchers with a comprehensive and transparent evidence-based overview of the characteristics of literature regarding initiatives of robot technology in dentistry. Data: All articles in which robot technology in dentistry is described, except for non-scientific articles and articles containing secondary data (reviews). Amongst others, the following data were extracted: type of study, level of technological readiness, authors’ professional background and the subject of interaction with the robot. Sources: Bibliographic databases PubMed, Embase, and Scopus were surveyed. A reference search was conducted. The search timeline was between January 1985 and October 2020. Study selection: A total of 911 articles were screened on title and abstract of which 161 deemed eligible for inclusion. Another 71 articles were excluded mainly because of unavailability of full texts or the sole use of secondary data (reviews). Four articles were included after hand searching the reference lists. In total, 94 articles were included for analysis. Conclusions: Since 2013 an average of six articles per year concern robot initiatives in dentistry, mostly originating from East Asia (57%). The vast majority of research was categorized as either basic theoretical or basic applied research (80%). Technology readiness levels did not reach higher than three (proof of concept) in 55% of all articles. In 84%, the first author of the included articles had a technical background and in 36%, none of the authors had a dental or medical background. The overall quality of literature, especially in terms of clinical validation, should be considered as low.

Deep Reinforcement Learning (DRL) has become a powerful strategy to solve complex decision making problems based on Deep Neural Networks (DNNs). However, it is highly data demanding, so unfeasible in physical systems for most applications. In this work, we approach an alternative Interactive Machine Learning (IML) strategy for training DNN policies based on human corrective feedback, with a method called Deep COACH (D-COACH). This approach not only takes advantage of the knowledge and insights of human teachers as well as the power of DNNs, but also has no need of a reward function (which sometimes implies the need of external perception for computing rewards). We combine Deep Learning with the COrrective Advice Communicated by Humans (COACH) framework, in which non-expert humans shape policies by correcting the agent’s actions during execution. The D-COACH framework has the potential to solve complex problems without much data or time required. Experimental results validated the efficiency of the framework in three different problems (two simulated, one with a real robot), with state spaces of low and high dimensions, showing the capacity to successfully learn policies for continuous action spaces like in the Car Racing and Cart-Pole problems faster than with DRL.

This paper presents a system for cooperatively manipulating large objects between a human and a robot. This physical interaction system is designed to handle, transport, or manipulate large objects of different shapes in cooperation with a human. Unique points are the bi-manual physical cooperation, the sequential characteristic of the cooperation including contact changes, and a novel architecture combining force interaction cues, interactive search-based planning, and online trajectory and motion generation. The resulting system implements a mixed initiative collaboration strategy, deferring to the human when his intentions are unclear, and driving the task once understood. This results in an easy and intuitive human-robot interaction. It is evaluated in simulations and on a bi-manual mobile robot with 32 degrees of freedom.

The work was supported by the grant project TK 01030108 - NEMENUS of the Technology Agency of the Czech Republic within the institutional support RVO: 61388998. Mechanical metamaterials are artificially produced structures that derive their properties from their periodically repeated structure rather than from mechanical properties of their base material. Often, these structures are manufactured by 3D printing. Nowadays the additive manufacturing technology can be used for producing complex bodies with complicated shapes and various properties which cannot be produced by conventional technologies. Such structures can posses superior properties such as e.g. the negative Poisson?s ratio. These materials exhibit a counter-intuitive behaviour when under the uniaxial tension, the structure expands transversely and vice versa. Applications of 3D printed bodies can be found in mechanical, biomechanical or aerospace engineering.

Recent years have seen a growing interest in the use of deep neural networks as function approximators in reinforcement learning. In this paper, an experience replay method is proposed that ensures that the distribution of the experiences used for training is between that of the policy and a uniform distribution. Through experiments on a magnetic manipulation task it is shown that the method reduces the need for sustained exhaustive exploration during learning. This makes it attractive in scenarios where sustained exploration is in-feasible or undesirable, such as for physical systems like robots and for life long learning. The method is also shown to improve the generalization performance of the trained policy, which can make it attractive for transfer learning. Finally, for small experience databases the method performs favorably when compared to the recently proposed alternative of using the temporal difference error to determine the experience sample distribution, which makes it an attractive option for robots with limited memory capacity.

Deep Neural Networks (DNNs) can be used as function approximators in Reinforcement Learning (RL). One advantage of DNNs is that they can cope with large input dimensions. Instead of relying on feature engineering to lower the input dimension, DNNs can extract the features from raw observations. The drawback of this end-to-end learning is that it usually requires a large amount of data, which for real-world control applications is not always available. In this paper, a new algorithm, Model Learning Deep Deterministic Policy Gradient (ML-DDPG), is proposed that combines RL with state representation learning, i.e., learning a mapping from an input vector to a state before solving the RL task. The ML-DDPG algorithm uses a concept we call predictive priors to learn a model network which is subsequently used to pre-train the first layer of the actor and critic networks. Simulation results show that the ML-DDPG can learn reasonable continuous control policies from high-dimensional observations that contain also task-irrelevant information. Furthermore, in some cases, this approach significantly improves the final performance in comparison to end-to-end learning.

Introduction: Patients with third degree burn wounds would benefit from a three-layer skin graft consisting of the epidermis, dermis and hypodermis. As part of the ArtiVasc 3D EU project we are trying to come closer to this goal and to generate a vascularized 3D skin substitute. Materials and methods: To generate the hypodermis a construct of hydrogels and electrospun matrices will be produced and then seeded with adipose derived stem cells (ASCs) and adipocytes. Results: ASCs adhered and proliferated on the matrices. After 21 days fat vacuoles could be detected as a sign of successful differentiation of ASCs. The use of bioreactors additionally improved the viability of cells within the hydrogel. Discussion and conclusions: The combination of hydrogels and electrospun matrices represents a promising approach for the generation of adipose tissue. Acknowledgments: The research was funded by the European Union Seventh Framework Programme (FP7) through the ArtiVasc 3D project under grant agreement no. 263416. Disclosures: The authors have nothing to disclose.

Efficient acquisition of new motor skills is among the most important abilities in order to make robot application more flexible, reduce the amount and cost of human programming as well as to make future robots more autonomous. However, most machine learning approaches to date are not capable to meet this challenge as they do not scale into the domain of high dimensional anthropomorphic and service robots. Instead, robot skill learning needs to rely upon task-appropriate approaches and domain insights. A particularly powerful approach has been driven by the concept of re-usable motor primitives. These have been used to learn a variety of “elementary movements” such as striking movements (e.g., hitting a T-ball, striking a table tennis ball), rhythmic movements (e.g., drumming, gaits for legged locomotion, padlling balls on a string), grasping, jumping and many others. Here, we take the approach to the next level and show experimentally how most elements required for table tennis can be addressed using motor primitives. We show four important components: (i) We present a motor primitive formulation that can deal with hitting and striking movements. (ii) We show how these can be initialized by imitation learning and (iii) generalized by reinforcement learning. (iv) We show how selection, generalization and pruning for motor primitives can be dealt with using a mixture of motor primitives. The resulting experimental prototypes can be shown to work well in practice.

Fragestellung: Lipofilling ist eine weit verbreitete Methode in der Plastischen und Rekonstruktiven Chirurgie. Der Erfolg dieser Methode hängt maßgeblich von den Resorptionsraten des Fettgewebes ab. Aus diesem Grund beschäftigen sich viele Studien mit der Identifizierung der Faktoren,[for full text, please go to the a.m. URL], 45. Jahrestagung der Deutschen Gesellschaft der Plastischen, Rekonstruktiven und Ästhetischen Chirurgen (DGPRÄC), 19. Jahrestagung der Vereinigung der Deutschen Ästhetisch-Plastischen Chirurgen (VDÄPC), 52. Jahrestagung der Österreichischen Gesellschaft für Plastische, Ästhetische und Rekonstruktive Chirurgie (ÖGPRÄC)

Hintergrund: Patienten mit drittgradigen Verbrennungen würden von einem dreischichtigen Hautersatz bestehend aus Epidermis, Dermis und Hypodermis profitieren. Im Rahmen des ArtiVasc3D EU-Projektes wird versucht diesem Ziel näher zu kommen und einen vaskularisierten Hautersatz zu generieren.[for full text, please go to the a.m. URL], 32. Jahrestagung der Deutschsprachigen Arbeitsgemeinschaft für Verbrennungsbehandlung (DAV 2014)

Einleitung: Die autologe Fettgewebstransplantation ist eine weit verbreitete Methode zur Weichteilaugmentation. Unterschiedliche Absaugsysteme zur Gewinnung des Fettgewebes sind hierzu mittlerweile erhältlich. Ein wichtiges Kriterium bei der Wahl des geeigneten Gerätes sollte die Qualität[for full text, please go to the a.m. URL], 43. Jahrestagung der Deutschen Gesellschaft der Plastischen, Rekonstruktiven und Ästhetischen Chirurgen (DGPRÄC), 17. Jahrestagung der Vereinigung der Deutschen Ästhetisch-Plastischen Chirurgen (VDÄPC)

Learning robots that can acquire new motor skills and refine existing ones have been a long standing vision of robotics, artificial intelligence, and the cognitive sciences. Early steps towards this goal in the 1980s made clear that reasoning and human insights will not suffice. Instead, new hope has been offered by the rise of modern machine learning approaches. However, to date, it becomes increasingly clear that off-the-shelf machine learning approaches will not be adequate for robot skill learning as these methods often do not scale into the high-dimensional domains of manipulator and humanoid robotics, nor do they fulfill the real-time requirement of the domain. As an alternative, we propose to divide the generic skill learning problem into parts that can be well-understood from a robotics point of view. After designing appropriate learning approaches for these basic components, these will serve as the ingredients of a general approach to robot skill learning. In this paper, we discuss our recent and current progress in this direction. As such, we present our work on learning to control, learning elementary movements, as well as our steps towards the learning of complex tasks. We show several evaluations using both real robots as well as physically realistic simulations.

Motor primitives or motion templates have become an important concept for both modeling human motor control as well as generating robot behaviors using imitation learning. Recent impressive results range from humanoid robot movement generation to timing models of human motions. The automatic generation of skill libraries containing multiple motion templates is an important step in robot learning. Such a skill learning system needs to cluster similar movements together and represent each resulting motion template as a generative model which is subsequently used for the execution of the behavior by a robot system. In this paper, we show how human trajectories captured as multidimensional time-series can be clustered using Bayesian mixtures of linear Gaussian state-space models based on the similarity of their dynamics. The appropriate number of templates is automatically determined by enforcing a parsimonious parametrization. As the resulting model is intractable, we introduce a novel approximation method based on variational Bayes, which is especially designed to enable the use of efficient inference algorithms. On recorded human Balero movements, this method is not only capable of finding reasonable motion templates but also yields a generative model which works well in the execution of this complex task on a simulated anthropomorphic SARCOS arm.

Trauergedicht auf Klaus Ernst von Platen, geheimer u. Regierungsrat im Herzogtum Magdeburg, Erbherr auf Dömmertin, +14. Aug. 1733

International audience; Deformable object manipulation (DOM) is an emerging research problem in robotics. The ability to manipulate deformable objects endows robots with higher autonomy and promises new applications in the industrial, services, and healthcare sectors. However, compared to rigid object manipulation, the manipulation of deformable objects is considerably more complex, and is still an open research problem. Addressing DOM challenges demand breakthroughs in almost all aspects of robotics, namely hardware design, sensing, (deformation) modeling, planning, and control. In this article, we review recent advances and highlight the main challenges when considering deformation in each sub-field. A particular focus of our paper lies in the discussions of these challenges and proposing future directions of research.

The arrival of intelligent, general-purpose robots that can learn to perform new tasks autonomously has been promised for a long time now. Deep reinforcement learning, which combines reinforcement learning with deep neural network function approximation, has the potential to enable robots to learn to perform a wide range of new tasks while requiring very little prior knowledge or human help. This framework might therefore help to finally make general purpose robots a reality. However, the biggest successes of deep reinforcement learning have so far been in simulated game settings. To translate these successes to the real world, significant improvements are needed in the ability of these methods to learn quickly and safely. This thesis investigates what is needed to make this possible and makes contributions towards this goal. Before deep reinforcement learning methods can be successfully applied in the robotics domain, an understanding is needed of how, when, and why deep learning and reinforcement learning work well together. This thesis therefore starts with a literature review, which is presented in Chapter 2. While the field is still in some regards in its infancy, it can already be noted that there are important components that are shared by successful algorithms. These components help to reconcile the differences between classical reinforcement learning methods and the training procedures used to successfully train deep neural networks. The main challenges in combining deep learning with reinforcement learning center around the interdependencies of the policy, the training data, and the training targets. Commonly used tools for managing the detrimental effects caused by these interdependencies include target networks, trust region updates, and experience replay buffers. Besides reviewing these components, a number of the more popular and historically relevant deep reinforcement learning methods are discussed.Reinforcement learning involves learning through trial and error. However, robots (and their surroundings) are fragile, which makes these trials---and especially errors---very costly. Therefore, the amount of exploration that is performed will often need to be drastically reduced over time, especially once a reasonable behavior has already been found. We demonstrate how, using common experience replay techniques, this can quickly lead to forgetting previously learned successful behaviors. This problem is investigated in Chapter 3. Experiments are conducted to investigate what distribution of the experiences over the state-action space leads to desirable learning behavior and what distributions can cause problems. It is shown how actor-critic algorithms are especially sensitive to the lack of diversity in the action space that can result form reducing the amount of exploration over time. Further relations between the properties of the control problem at hand and the required data distributions are also shown. These include a larger need for diversity in the action space when control frequencies are high and a reduced importance of data diversity for problems where generalizing the control strategy across the state-space is more difficult.While Chapter 3 investigates what data distributions are most beneficial, Chapter 4 instead proposes practical algorithms to {select} useful experiences from a stream of experiences. We do not assume to have any control over the stream of experiences, which makes it possible to learn from additional sources of experience like other robots, experiences obtained while learning different tasks, and experiences obtained using predefined controllers. We make two separate judgments on the utility of individual experiences. The first judgment is on the long term utility of experiences, which is used to determine which experiences to keep in memory once the experience buffer is full. The second judgment is on the instantaneous utility of the experience to the learning agent. This judgment is used to determine which experiences should be sampled from the buffer to be learned from. To estimate the short and long term utility of the experiences we propose proxies based on the age, surprise, and the exploration intensity associated with the experiences. It is shown how prior knowledge of the control problem at hand can be used to decide which proxies to use. We additionally show how the knowledge of the control problem can be used to estimate the optimal size of the experience buffer and whether or not to use importance sampling to compensate for the bias introduced by the selection procedure. Together, these choices can lead to a more stable learning procedure and better performing controllers. In Chapter 5 we look at what to learn form the collected data. The high price of data in the robotics domain makes it crucial to extract as much knowledge as possible from each and every datum. Reinforcement learning, by default, does not do so. We therefore supplement reinforcement learning with explicit state representation learning objectives. These objectives are based on the assumption that the neural network controller that is to be learned can be seen as consisting of two consecutive parts. The first part (referred to as the state encoder) maps the observed sensor data to a compact and concise representation of the state of the robot and its environment. The second part determines which actions to take based on this state representation. As the representation of the state of the world is useful for more than just completing the task at hand, it can also be trained with more general (state representation learning) objectives than just the reinforcement learning objective associated with the current task. We show how including these additional training objectives allows for learning a much more general state representation, which in turn makes it possible to learn broadly applicable control strategies more quickly. We also introduce a training method that ensures that the added learning objectives further the goal of reinforcement learning, without destabilizing the learning process through their changes to the state encoder. The final contribution of this thesis, presented in Chapter 6, focuses on the optimization procedure used to train the second part of the policy; the mapping from the state representation to the actions. While we show that the state encoder can be efficiently trained with standard gradient-based optimization techniques, perfecting this second mapping is more difficult. Obtaining high quality estimates of the gradients of the policy performance with respect to the parameters of this part of the neural network is usually not feasible. This means that while a reasonable policy can be obtained relatively quickly using gradient-based optimization approaches, this speed comes at the cost of the stability of the learning process as well as the final performance of the controller. Additionally, the unstable nature of this learning process brings with it an extreme sensitivity to the values of the hyper-parameters of the training method. This places an unfortunate emphasis on hyper-parameter tuning for getting deep reinforcement learning algorithms to work well. Gradient-free optimization algorithms can be more simple and stable, but tend to be much less sample efficient. We show how the desirable aspects of both methods can be combined by first training the entire network through gradient-based optimization and subsequently fine-tuning the final part of the network in a gradient-free manner. We demonstrate how this enables the policy to improve in a stable manner to a performance level not obtained by gradient-based optimization alone, using many fewer trials than methods using only gradient-free optimization.

Background— This large, prospective, multicentric study was performed to analyze the distribution of tricuspid regurgitation velocity (TRV) values during exercise and hypoxia in relatives of patients with idiopathic and familial pulmonary arterial hypertension (PAH) and in healthy control subjects. We tested the hypothesis that relatives of idiopathic/familial PAH patients display an enhanced frequency of hypertensive TRV response to stress and that this response is associated with mutations in the bone morphogenetic protein receptor II ( BMPR2 ) gene. Methods and Results— TRV was estimated by Doppler echocardiography during supine bicycle exercise in normoxia and during 120 minutes of normobaric hypoxia (F io 2 =12%; ≈4500 m) in 291 relatives of 109 PAH patients and in 191 age-matched control subjects. Mean maximal TRVs were significantly higher in PAH relatives during both exercise and hypoxia. During exercise, 10% of control subjects but 31.6% of relatives ( P P =0.0029). Among control subjects, TRV at rest was not related to age, sex, body mass index, systemic blood pressure, smoking status, or heart rate. Within kindreds identified as harboring deleterious mutations of the BMPR2 gene, a hypertensive TRV response occurred significantly more often compared with those without detected mutations. Conclusions— Pulmonary hypertensive response to exercise and hypoxia in idiopathic/familial PAH relatives appears as a genetic trait with familial clustering, being correlated to but not caused by a BMPR2 mutation. The suitability of this trait to predict manifest PAH development should be addressed in long-term follow-up studies.