Search

Sim-to-real Deep Reinforcement Learning (DRL) has shown promising in subtasks automation for surgical robotic systems, since it allows to safely perform all the trial and error attempts needed to learn the optimal control policy. However, a realistic simulation environment is essential to guarantee direct transfer of the learnt policy from the simulated to the real system. In this work, we introduce UnityFlexML, an open-source framework providing support for soft bodies simulation and state-of-the-art DRL methods. We demonstrate that a DRL agent can be successfully trained within UnityFlexML to manipulate deformable fat tissues for tumor exposure during a nephrectomy procedure. Furthermore, we show that the learned policy can be directly deployed on the da Vinci Research Kit, which is able to execute the trajectories generated by the DRL agent. The proposed framework represents an essential component for the development of autonomous robotic systems, where the interaction with the deformable anatomical environment is involved., https://youtu.be/qrKSB3N32T0

International audience

Realistic and fast simulation of anatomical deforma- tions due to ultrasound probe pressure is of outstanding impor- tance for testing and validation of autonomous robotic ultrasound systems. We propose a deformation model which relies on the position-based dynamics (PBD) approach to simulate the probe- tissue interaction and predict the displacement of internal targets during US acquisition. Performances of the patient-specific PBD anatomical model are evaluated in comparison to two different simulations relying on the traditional finite element (FE) method, in the context of breast ultrasound scanning. Localization error obtained when applying the PBD model remains below 11 mm for all the tumors even for input displacements in the order of 30 mm. The proposed method is able to achieve a better trade-off among accuracy, computation time and generalization capabilities with respect to the two FE models. Position-based dynamics approach has proved to be successful in modeling breast tissue deformations during US acquisition. It represents a valid alternative to classical FE methods for simulating the interaction between US probe and tissues., https://youtu.be/cAlOJ4JZKt8

Fusion-based ultrasound (US)-guided biopsy in a breast is challenging due to the high deformability of the tissue combined with the fact that the breast is usually differently deformed in CT, MR, and US acquisition which makes registration difficult. With this phantom study, we demonstrate the feasibility of a fusion-based ultrasound-guided method for breast biopsy. 3D US and 3D CT data were acquired using dedicated imaging setups of a breast phantom freely hanging in prone position with lesions. The 3D breast CT set up was provided by Koning (Koning Corp., West Henrietta, NY). For US imaging, a dedicated breast scanning set up was developed consisting of a cone-shaped revolving water tank with a 152- mm-sized US transducer mounted in its wall and an aperture for needle insertion. With this setup, volumetric breast US data (0.5×0.5×0.5 mm3 voxel size) can be collected and reconstructed within 3 minutes. The position of the lesion as detected with breast CT was localized in the US data by rigid registration. After lesion localization, the tank rotates the transducer until the lesion is in the US plane. Since the lesion was visible on ultrasound, the performance of the registration was validated. To facilitate guided biopsy, the lesion motion, induced by needle insertion, is estimated using cross-correlation-based speckle tracking and the tracked lesion visualized in the US image at an update frequency of 10 Hz. Thus, in conclusion a fusion-based ultrasound-guided method was introduced which enables ultrasound-guided biopsy in breast that is applicable also for ultrasound occult lesions.

Autonomy in robot-assisted surgery is essential to reduce surgeons' cognitive load and eventually improve the overall surgical outcome. A key requirement for autonomy in a safety-critical scenario as surgery lies in the generation of interpretable plans that rely on expert knowledge. Moreover, the Autonomous Robotic Surgical System (ARSS) must be able to reason on the dynamic and unpredictable anatomical environment, and quickly adapt the surgical plan in case of unexpected situations. In this paper, we present a modular Framework for Robot-Assisted Surgery (FRAS) in deformable anatomical environments. Our framework integrates a logic module for task-level interpretable reasoning, a biomechanical simulation that complements data from real sensors, and a situation awareness module for context interpretation. The framework performance is evaluated on simulated soft tissue retraction, a common surgical task to remove the tissue hiding a region of interest. Results show that the framework has the adaptability required to successfully accomplish the task, handling dynamic environmental conditions and possible failures, while guaranteeing the computational efficiency required in a real surgical scenario. The framework is made publicly available., Comment: 2021 IEEE International Symposium on Medical Robotics (ISMR)