Search

Many individuals with movement disorders are unable to make efficient use of graphical computer interfaces commonly employed in personal computers. In this study, performance limitations in target tracking with a computer mouse were studied for eight young subjects aged 19 to 29 (M=23, SD=3), four old subjects aged 70 to 73 (M=72, SD=1), and five motor-disabled subjects aged 37 to 74 (M=65, SD=16). Subjects tracked simple one- and two-dimensional motions at various frequencies. Performance was measured using an accuracy index derived from root-mean-square error, and a linearity index based on coherence estimation. A maximum bandwidth of 2 Hz for accuracy of mouse use was found, which often decreased due to advanced age or motor disability. Tracking linearity of all groups decreased as frequency increased. A significant degree of nonlinearity existed in all results (p<0.05), with disabled subjects nearing complete nonlinearity in two-dimensional tracking. The data show that with advanced age and disability, mouse use becomes increasingly inaccurate and nonlinear. Assistive computer interfacing techniques, such as signal filtering, may improve mouse use. [ABSTRACT FROM AUTHOR]

HeartPrinter is a novel under-constrained 3-cable parallel wire robot designed for minimally invasive epicardial interventions. The robot adheres to the beating heart using vacuum suction at its anchor points, with a central injector head that operates within the triangular workspace formed by the anchors, and is actuated by cables for multipoint direct gene therapy injections. Minimizing cable tensions can reduce forces on the heart at the anchor points while supporting rapid delivery of accurate injections and minimizing procedure time, risk of damage to the robot, and strain to the heart. However, cable tensions must be sufficient to hold the injector head's position as the heart moves and to prevent excessive cable slack. We pose a linear optimization problem to minimize the sum of cable tension magnitudes for HeartPrinter while ensuring the injector head is held in static equilibrium and the tensions are constrained within a feasible range. We use Karush-Kuhn-Tucker optimality conditions to derive conditional algebraic expressions for optimal cable tensions as a function of injector head position and workspace geometry, and we identify regions of injector head positions where particular combinations of cable tensions are optimally at minimum allowable tensions. The approach can rapidly solve for the minimum set of cable tensions for any robot workspace geometry and injector head position and determine whether an injection site is attainable., (Copyright © 2023 by ASME.)

Intraocular surgery, one of the most challenging discipline of microsurgery, requires sensory and motor skills at the limits of human physiological capabilities combined with tremendously difficult requirements for accuracy and steadiness. Nowadays, robotics combined with advanced imaging has opened conspicuous and significant directions in advancing the field of intraocular microsurgery. Having patient treatment with greater safety and efficiency as the final goal, similar to other medical applications, robotics has a real potential to fundamentally change microsurgery by combining human strengths with computer and sensor-based technology in an information-driven environment. Still in its early stages, robotic assistance for intraocular microsurgery has been accepted with precaution in the operating room and successfully tested in a limited number of clinical trials. However, owing to its demonstrated capabilities including hand tremor reduction, haptic feedback, steadiness, enhanced dexterity, micrometer-scale accuracy, and others, microsurgery robotics has evolved as a very promising trend in advancing retinal surgery. This paper will analyze the advances in retinal robotic microsurgery, its current drawbacks and limitations, as well as the possible new directions to expand retinal microsurgery to techniques currently beyond human boundaries or infeasible without robotics.

Background: HeartLander is a tethered robot walker that utilizes suction to adhere to the beating heart. HeartLander can be used for minimally invasive administration of cardiac medications or ablation of tissue. In order to administer injections safely, HeartLander must avoid coronary vasculature., Methods: Doppler ultrasound signals were recorded using a custom-made cardiac phantom and used to classify different coronary vessel properties. The classification was performed by two machine learning algorithms, the support vector machines and a deep convolutional neural network. These algorithms were then validated in animal trials., Results: Accuracy of identifying vessels above turbulent flow reached greater than 92% in phantom trials and greater than 98% in animal trials., Conclusions: Through the use of two machine learning algorithms, HeartLander has shown the ability to identify different sized vasculature proximally above turbulent flow. These results indicate that it is feasible to use Doppler ultrasound to identify and avoid coronary vasculature during cardiac interventions using HeartLander., (© 2021 John Wiley & Sons Ltd.)

Background: Cardiac gene therapies lack effective delivery methods to the myocardium. While direct injection has demonstrated success over a small region, homogenous gene expression requires many injections over a large area. To address this need, we developed a minimally invasive flexible parallel wire robot for epicardial interventions. To accurately deploy it onto the beating heart, an introducer mechanism is required., Methods: Two mechanisms are presented. Assessment of the robot's positioning, procedure time, and pericardium insertion forces are performed on an artificial beating heart., Results: Successful positioning was demonstrated. The mean procedure time was 230 ± 7 seconds for mechanism I and 259 ± 4 seconds for mechanism II. The mean pericardium insertion force was 2.2 ± 0.4 N anteriorly and 3.1 ± 0.4 N posteriorly., Conclusion: Introducer mechanisms demonstrate feasibility in facilitating the robot's deployment on the epicardium. Pericardium insertion forces and procedure times are consistent and reasonable., Competing Interests: Professor Cameron N Riviere reports holding equity in HeartLander Surgical, Inc; In addition, Professor Cameron N Riviere has a patent US10736703B2 issued to Carnegie Mellon University. The authors report no other conflicts of interest in this work., (© 2021 Ladak et al.)

Retinal membrane peeling requires delicate manipulation. The presence of the surgeon's physiological tremor, the high variability and often low quality of the ophthalmic image, and excessive forces make the tasks more challenging. Preventing unintended movement caused by tremor and unintentional forces can reduce membrane injury. With the use of an actively stabilized handheld robot, we employ a monocular camera-based surface reconstruction method to estimate the retinal plane and we propose the use of a virtual fixture with the application of a hard stop and motion scaling to improve control of the tool tip during delaminating in a laboratory simulation of retinal membrane peeling. A hard stop helps to limit downward force exerted on the surface. Motion scaling also improves the user's control of contact force when delaminating. We demonstrate a reduction of maximum force and maximum surface-penetration distance from the estimated retinal plane using the proposed technique., (Copyright © 2021 by ASME.)

Recently, there has been active research in finding robotized solutions for the treatment of atrial fibrillation (AF) by augmenting catheter systems through the integration of force sensors at the tip. However, limited research has been aimed at providing automatic force control by also integrating actuation of the catheter tip, which can significantly enhance safety in such procedures. This article solves the demanding challenge of miniaturizing both actuation and sensing for integration into flexible catheters. Fabrication strategies are presented for a series of novel soft thick-walled cylindrical actuators, with embedded sensing using eutectic gallium-indium. The functional catheter tips have a diameter in the range of 2.6-3.6 mm and can both generate and detect forces in the range of < 0.4 N, with a bandwidth of 1-2 Hz. The deformation modeling of thick-walled cylinders with fiber reinforcement is presented in the article. An experimental setup developed for static and dynamic characterization of these units is presented. The prototyped units were validated with respect to the design specifications. The preliminary force control results indicate that these units can be used in tracking and control of contact force, which has the potential to make AF procedures much safer and more accurate.

Objective: Intraoperative neurophysiological monitoring (IONM) is the use of electrophysiological methods during certain high-risk surgeries to assess the functional integrity of nerves in real time and alert the surgeon to prevent damage. However, the efficiency of IONM in current practice is limited by latency of verbal communications, inter-rater variability, and the subjective manner in which electrophysiological signals are described., Methods: In an attempt to address these shortcomings, we investigate automated classification of free-running electromyogram (EMG) waveforms during IONM. We propose a hybrid model with a convolutional neural network (CNN) component and a long short-term memory (LSTM) component to better capture complicated EMG patterns under conditions of both electrical noise and movement artifacts. Moreover, a preprocessing pipeline based on data normalization is used to handle classification of data from multiple subjects. To investigate model robustness, we also analyze models under different methods for processing of artifacts., Results: Compared with several benchmark modeling methods, CNN-LSTM performs best in classification, achieving accuracy of 89.54% and sensitivity of 94.23% in cross-patient evaluation., Conclusion: The CNN-LSTM model shows promise for automated classification of continuous EMG in IONM., Significance: This technique has potential to improve surgical safety by reducing cognitive load and inter-rater variability.

Purpose: Robot-assisted intraocular microsurgery can improve performance by aiding the surgeon in operating on delicate micron-scale anatomical structures of the eye. In order to account for the eyeball motion that is typical in intraocular surgery, there is a need for fast and accurate algorithms that map the retinal vasculature and localize the retina with respect to the microscope., Methods: This work extends our previous work by a graph-based SLAM formulation using a sparse incremental smoothing and mapping (iSAM) algorithm., Results: The resulting technique, "EyeSAM," performs SLAM for intraoperative vitreoretinal surgical use while avoiding spurious duplication of structures as with the previous simpler technique. The technique also yields reduction in average pixel error in the camera motion estimation., Conclusions: This work provides techniques to improve intraoperative tracking of retinal vasculature by handling loop closures and achieving increased robustness to quick shaky motions and drift due to uncertainties in the motion estimation.

Background: Organ-mounted robots adhere to the surface of a mobile organ as a platform for minimally invasive interventions, providing passive compensation of physiological motion. This approach is beneficial during surgery on the beating heart. Accurate localization in such applications requires accounting for the heartbeat and respiratory motion. Previous work has described methods for modeling quasi-periodic motion of a point and registering to a static preoperative map. The existing techniques, while accurate, require several respiratory cycles to converge., Methods: This paper presents a general localization technique for this application, involving function approximation using radial basis function (RBF) interpolation., Results: In an experiment in the porcine model in vivo, the technique yields mean localization accuracy of 1.25 mm with a 95% confidence interval of 0.22 mm., Conclusions: The RBF approximation provides accurate estimates of robot location instantaneously., (© 2018 John Wiley & Sons, Ltd.)

Microsurgical procedures, such as petroclival meningioma resection, require careful surgical actions in order to remove tumor tissue, while avoiding brain and vessel damaging. Such procedures are currently performed under microscope magnification. Robotic tools are emerging in order to filter surgeons' unintended movements and prevent tools from entering forbidden regions such as vascular structures. The present work investigates the use of a handheld robotic tool (Micron) to automate vessel avoidance in microsurgery. In particular, we focused on vessel segmentation, implementing a deep-learning-based segmentation strategy in microscopy images, and its integration with a feature-based passive 3D reconstruction algorithm to obtain accurate and robust vessel position. We then implemented a virtual-fixture-based strategy to control the handheld robotic tool and perform vessel avoidance. Clay vascular phantoms, lying on a background obtained from microscopy images recorded during petroclival meningioma surgery, were used for testing the segmentation and control algorithms. When testing the segmentation algorithm on 100 different phantom images, a median Dice similarity coefficient equal to 0.96 was achieved. A set of 25 Micron trials of 80 s in duration, each involving the interaction of Micron with a different vascular phantom, were recorded, with a safety distance equal to 2 mm, which was comparable to the median vessel diameter. Micron's tip entered the forbidden region 24% of the time when the control algorithm was active. However, the median penetration depth was 16.9 μm, which was two orders of magnitude lower than median vessel diameter. Results suggest the system can assist surgeons in performing safe vessel avoidance during neurosurgical procedures.

Background: Organ-mounted robots address the problem of beating-heart surgery by adhering to the heart, passively providing a platform that approaches zero relative motion. Because of the quasi-periodic deformation of the heart due to heartbeat and respiration, registration must address not only spatial registration but also temporal registration., Methods: Motion data were collected in the porcine model in vivo (N = 6). Fourier series models of heart motion were developed. By comparing registrations generated using an iterative closest-point approach at different phases of respiration, the phase corresponding to minimum registration distance is identified., Results: The spatiotemporal registration technique presented here reduces registration error by an average of 4.2 mm over the 6 trials, in comparison with a more simplistic static registration that merely averages out the physiological motion., Conclusions: An empirical metric for spatiotemporal registration of organ-mounted robots is defined and demonstrated using data from animal models in vivo., (Copyright © 2018 John Wiley & Sons, Ltd.)

This paper presents techniques for robot-aided intraocular surgery using monocular vision in order to overcome erroneous stereo reconstruction in an intact eye. We propose a new retinal surface estimation method based on a structured-light approach. A handheld robot known as the Micron enables automatic scanning of a laser probe, creating projected beam patterns on the retinal surface. Geometric analysis of the patterns then allows planar reconstruction of the surface. To realize automated surgery in an intact eye, monocular hybrid visual servoing is accomplished through a scheme that incorporates surface reconstruction and partitioned visual servoing. We investigate the sensitivity of the estimation method according to relevant parameters and also evaluate its performance in both dry and wet conditions. The approach is validated through experiments for automated laser photocoagulation in a realistic eye phantom in vitro. Finally, we present the first demonstration of automated intraocular laser surgery in porcine eyes ex vivo.

Each year 35,000 cardiac ablation procedures are performed to treat atrial fibrillation through the use of catheter systems. The success rate of this treatment is highly dependent on the force which the catheter applies on the heart wall. If the magnitude of the applied force is much higher than a certain threshold the tissue perforates, whereas if the force is lower than this threshold the lesion size may be too large and is inconsistent. Furthermore, studies have shown large variability in the applied force from trained physicians during treatment, suggesting that physicians are unable to manually regulate the levels of the force at the site of treatment. Current catheter systems do not provide the physicians with active means for contact force control and are only at most aided by visual feedback of the forces measured in situ. This paper discusses a novel design of a robotic end-effector that integrates mechanisms of sensing and actively controlling of the applied forces into a miniaturized compact form. The required specifications for design and integration were derived from the current application under investigation. An off-the-shelf miniature piezoelectric motor was chosen for actuation, and a force sensing solution was developed to meet the specifications. Experimental characterization of the actuator and the force sensor within the integrated setup show compliance with the specifications and pave the way for future experimentation where closed-loop control of the system can be implemented according to the contact force control strategies for the application.

Background: Fast and accurate mapping and localization of the retinal vasculature is critical to increasing the effectiveness and clinical utility of robot-assisted intraocular microsurgery such as laser photocoagulation and retinal vessel cannulation., Methods: The proposed EyeSLAM algorithm delivers 30 Hz real-time simultaneous localization and mapping of the human retina and vasculature during intraocular surgery, combining fast vessel detection with 2D scan-matching techniques to build and localize a probabilistic map of the vasculature., Results: In the harsh imaging environment of retinal surgery with high magnification, quick shaky motions, textureless retina background, variable lighting and tool occlusion, EyeSLAM can map 75% of the vessels within two seconds of initialization and localize the retina in real time with a root mean squared (RMS) error of under 5.0 pixels (translation) and 1° (rotation)., Conclusions: EyeSLAM robustly provides retinal maps and registration that enable intelligent surgical micromanipulators to aid surgeons in simulated retinal vessel tracing and photocoagulation tasks., (Copyright © 2017 John Wiley & Sons, Ltd.)

Background: Organ-mounted robots passively compensate heartbeat and respiratory motion. In model-guided procedures, this motion can be a significant source of information that can be used to aid in localization or to add dynamic information to static preoperative maps., Methods: Models for estimating periodic motion are proposed for both position and orientation. These models are then tested on animal data and optimal orders are identified. Finally, methods for online identification are demonstrated., Results: Models using exponential coordinates and Euler-angle parameterizations are as accurate as models using quaternion representations, yet require a quarter fewer parameters. Models which incorporate more than four cardiac or three respiration harmonics are no more accurate. Finally, online methods estimate model parameters as accurately as offline methods within three respiration cycles., Conclusions: These methods provide a complete framework for accurately modelling the periodic deformation of points anywhere on the surface of the heart in a closed chest., (Copyright © 2017 John Wiley & Sons, Ltd.)

Frequency domain multiplexing (FDM) is a useful for making multiple measurements simultaneously, such as in optical and electromagnetic position trackers. Much interference is periodic (e.g., AC power harmonics), and is rejected well by FDM, but impulse disturbances are also common. Impulses corrupt the entire spectrum for a short period, and are better rejected in the time domain. Nonlinear blanking is a simple way to suppress impulses, but cannot be easily realized when the required dynamic range is large, and problematic noise may be far smaller than the signal. The described multi-rate Kalman filter upsamples the prediction to the input rate so that impulse departures from the predicted signal are easily detected and blanked out. Also, noise levels in unused adjacent channels are used to estimate measurement noise so that the Kalman filter adapts more slowly when noise is high, keeping output noise roughly constant even in the presence of longer noise bursts.

In this paper we describe work towards retinal vessel cannulation using an actively stabilized handheld robot, guided by monocular vision. We employ a previously developed monocular camera based surface reconstruction method using automated laser beam scanning over the retina. We use the reconstructed plane to find a coordinate transform between the 2D image plane coordinate system and the global 3D frame. Within a hemispherical region around the target, we use motion scaling for higher precision. The contribution of this work is the homography matrix estimation using monocular vision and application of the previously developed laser surface reconstruction to Micron guided vein cannulation. Experiments are conducted in a wet eye phantom to show the higher accuracy of the surface reconstruction as compared to standard stereo reconstruction. Further, experiments to show the increased surgical accuracy due to motion scaling are also carried out.

Background: Present treatments for ventricular tachycardia have significant drawbacks. To ameliorate these drawbacks, it may be advantageous to employ an epicardial robotic walker that performs mapping and ablation with precise control of needle insertion depth. This paper examines the feasibility of such a system., Methods: This paper describes techniques for epicardial mapping and depth-controlled ablation with the robotic walker. The mapping technique developed for the current form of the system uses a single equivalent moving dipole model combined with the navigation capability of the walker. The intervention technique provides saline-enhanced radio frequency ablation, with sensing of needle penetration depth. The mapping technique was demonstrated in an artificial heart model with a simulated arrhythmia focus, followed by preliminary testing in the porcine model in vivo. The ablation technique was demonstrated in an artificial tissue model, and then in chicken breast tissue ex vivo., Results: The walker located targets to within 2 mm using the SEMDM technique. No epicardial damage was found subsequent to the porcine trial in vivo. Needle insertion for ablation was controlled to within 2 mm of the target depth. Lesion size was repeatable, with diameter varying consistently in proportion to volume of saline injected., Conclusions: The experiments demonstrated the general feasibility of the techniques for mapping and depth-controlled ablation with the robotic walker.

Background: Current practice in neurosurgical needle insertion is limited by the straight trajectories inherent with rigid probes. One technique allowing curvilinear trajectories involves flexible bevel-tipped needles, which bend during insertion due to their asymmetry. In the brain, safety will require avoidance of the sharp tips often used in laboratory studies, in favor of a more rounded profile. Steering performance, on the other hand, requires maximal asymmetry. Design of safe bevel-tipped brain needles thus involves management of this tradeoff by adjusting needle gauge, bevel angle, and fillet (or tip) radius to arrive at a design that is suitably asymmetrical while producing strain, strain rate, and stress below the levels that would damage brain tissue., Methods: Designs with a variety of values of needle radius, bevel angle, and fillet radius were evaluated in finite-element simulations of simultaneous insertion and rotation. Brain tissue was modeled as a hyperelastic, linear viscoelastic material. Based on the literature available, safety thresholds of 0.19 strain, 10 s -1 strain rate, and 120 kPa stress were used. Safe values of needle radius, bevel angle, and fillet radius were selected, along with an appropriate velocity envelope for safe operation. The resulting needle was fabricated and compared with a Sedan side-cutting brain biopsy needle in a study in the porcine model in vivo ( N =3)., Results: The prototype needle selected was 1.66 mm in diameter, with bevel angle of 10° and fillet radius of 0.25 mm. Upon examination of postoperative CT and histological images, no differences in tissue trauma or hemorrhage were noted between the prototype needle and the Sedan needle., Conclusions: The study indicates a general design technique for safe bevel-tipped brain needles based on comparison with relevant damage thresholds for strain, strain rate, and stress. The full potential of the technique awaits the determination of more exact safety thresholds., Competing Interests: Disclosure Authors reported no conflicts of interest.

We describe the development of the In-Loop Electromagnetic Tracker (ILEMT), designed to meet the demanding latency and resolution requirements for active stabilization of hand motion during precision manipulations such as microsurgery. The prototype surpasses the fastest commercial EM trackers by > 4× in root bandwidth/resolution and 2× in latency. The use of two widely spaced carrier frequencies (e.g., 300 Hz and 10 kHz) enables a particularly simple way of reducing the eddy-current interference caused by nonferrous metals present in the workspace. Previously, metal compatibility has only been achieved at a large cost to measurement speed.

Background: In preclinical testing, ventricular wall injection of hydrogels has been shown to be effective in modulating ventricular remodeling and preserving cardiac function. For some approaches, early-stage clinical trials are under way. The hydrogel delivery method varies, with minimally invasive approaches being preferred. Endocardial injections carry a risk of hydrogel regurgitation into the circulation, and precise injection patterning is a challenge. An epicardial approach with a thermally gelling hydrogel through the subxiphoid pathway overcomes these disadvantages., Methods: A relatively stiff, thermally responsive, injectable hydrogel based on N-isopropylacrylamide and N-vinylpyrrolidone (VP gel) was synthesized and characterized. VP gel thermal behavior was tuned to couple with a transepicardial injection robot, incorporating a cooling feature to achieve injectability. Ventricular wall injections of the optimized VP gel have been performed ex vivo and on beating porcine hearts., Results: Thermal transition temperature, viscosity, and gelling time for the VP gel were manipulated by altering N-vinylpyrrolidone content. The target parameters for cooling in the robotic system were chosen by thermal modeling to support smooth, repeated injections on an ex vivo heart. Injections at predefined locations and depth were confirmed in an infarcted porcine model., Conclusions: A coupled thermoresponsive hydrogel and robotic injection system incorporating a temperature-controlled injectate line was capable of targeted injections and amenable to use with a subxiphoid transepicardial approach for hydrogel injection after myocardial infarction. The confirmation of precise location and depth injections would facilitate a patient-specific planning strategy to optimize injection patterning to maximize the mechanical benefits of hydrogel placement., (Copyright © 2016 The Society of Thoracic Surgeons. Published by Elsevier Inc. All rights reserved.)

Gene therapies for heart failure have emerged in recent years, yet they lack an effective method for minimally invasive, uniform delivery. To address this need we developed a minimally invasive parallel wire robot for epicardial interventions. Accurate and safe interventions using this device require control of force in addition to injector position. Accounting for the nonidealities of the device design, however, yields nonlinear and underconstrained statics. This work solves these equations and demonstrates the efficacy of using this information in a parallel control scheme, which is shown to provide superior positioning compared to a position-only controller.

Background: Peeling procedures in retinal surgery require micron-scale manipulation and control of sub-tactile forces., Methods: Hybrid position/force control of an actuated handheld microsurgical instrument is presented as a means for simultaneously improving positioning accuracy and reducing forces to prevent avoidable trauma to tissue. The system response was evaluated, and membrane-peeling trials were performed by four test subjects in both artificial and animal models., Results: Maximum force was reduced by 56% in both models compared with position control. No statistically significant effect on procedure duration was observed., Conclusions: A hybrid position/force control system has been implemented that successfully attenuates forces and minimizes unwanted excursions during microsurgical procedures such as membrane peeling. Results also suggest that improvements in safety using this technique may be attained without increasing the duration of the procedure., (Copyright © 2015 John Wiley & Sons, Ltd.)

This paper presents robot-aided intraocular laser surgery using a handheld robot known as Micron. The micromanipulator incorporated in Micron enables visual servoing of a laser probe, while maintaining a constant distance of the tool tip from the retinal surface. The comparative study was conducted with various control methods for evaluation of robot-aided intraocular laser surgery.

Background and Objective: Laser photocoagulation is a mainstay or adjuvant treatment for a variety of common retinal diseases. Automated laser photocoagulation during intraocular surgery has not yet been established. The authors introduce an automated laser photocoagulation system for intraocular surgery, based on a novel handheld instrument. The goals of the system are to enhance accuracy and efficiency and improve safety., Materials and Methods: Triple-ring patterns are introduced as a typical arrangement for the treatment of proliferative retinopathy and registered to a preoperative fundus image. In total, 32 target locations are specified along the circumferences of three rings having diameters of 1, 2, and 3 mm, with a burn spacing of 600 μm. Given the initial system calibration, the retinal surface is reconstructed using stereo vision, and the targets specified on the preoperative image are registered with the control system. During automated operation, the laser probe attached to the manipulator of the active handheld instrument is deflected as needed via visual servoing in order to correct the error between the aiming beam and a specified target, regardless of any erroneous handle motion by the surgeon. A constant distance of the laser probe from the retinal surface is maintained in order to yield consistent size of burns and ensure safety during operation. Real-time tracking of anatomical features enables compensation for any movement of the eye. A graphical overlay system within operating microscope provides the surgeon with guidance cues for automated operation. Two retinal surgeons performed automated and manual trials in an artificial model of the eye, with each trial repeated three times. For the automated trials, various targeting thresholds (50-200 μm) were used to automatically trigger laser firing. In manual operation, fixed repetition rates were used, with frequencies of 1.0-2.5 Hz. The power of the 532 nm laser was set at 3.0 W with a duration of 20 ms. After completion of each trial, the speed of operation and placement error of burns were measured. The performance of the automated laser photocoagulation was compared with manual operation, using interpolated data for equivalent firing rates from 1.0 to 1.75 Hz., Results: In automated trials, average error increased from 45 ± 27 to 60 ± 37 μm as the targeting threshold varied from 50 to 200 μm, while average firing rate significantly increased from 0.69 to 1.71 Hz. The average error in the manual trials increased from 102 ± 67 to 174 ± 98 μm as firing rate increased from 1.0 to 2.5 Hz. Compared to the manual trials, the average error in the automated trials was reduced by 53.0-56.4%, resulting in statistically significant differences (P ≤ 10(-20) ) for all equivalent frequencies (1.0-1.75 Hz). The depth of the laser tip in the automated trials was consistently maintained within 18 ± 2 μm root-mean-square (RMS) of its initial position, whereas it significantly varied in the manual trials, yielding an error of 296 ± 30 μm RMS. At high firing rates in manual trials, such as at 2.5 Hz, laser photocoagulation is marginally attained, yielding failed burns of 30% over the entire pattern, whereas no failed burns are found in automated trials. Relatively regular burn sizes are attained in the automated trials by the depth servoing of the laser tip, while burn sizes in the manual trials vary considerably. Automated avoidance of blood vessels was also successfully demonstrated, utilizing the retina-tracking feature to identify avoidance zones., Conclusion: Automated intraocular laser surgery can improve the accuracy of photocoagulation while ensuring safety during operation. This paper provides an initial demonstration of the technique under reasonably realistic laboratory conditions; development of a clinically applicable system requires further work., (© 2015 Wiley Periodicals, Inc.)

Retinal vein cannulation is a demanding procedure proposed to treat retinal vein occlusion by direct therapeutic agent delivery methods. Challenges in identifying the moment of venous puncture, achieving cannulation and maintaining cannulation during drug delivery currently limit the feasibility of the procedure. In this study, we respond to these problems with an assistive system combining a handheld micromanipulator, Micron, with a force-sensing microneedle. The integrated system senses the instant of vein puncture based on measured forces and the position of the needle tip. The system actively holds the cannulation device securely in the vein following cannulation and during drug delivery. Preliminary testing of the system in a dry phantom, stretched vinyl membranes, demonstrates a significant improvement in the total time the needle could be maintained stably inside of the vein. This was especially evident in smaller veins and is attributed to decreased movement of the positioned cannula following venous cannulation.

Gene therapies have emerged as a promising treatment for congestive heart failure, yet they lack a method for minimally invasive, uniform delivery. To address this need we developed Cerberus, a minimally invasive parallel wire robot for cardiac interventions. Prior work on controlling the movement of Cerberus required accurate knowledge of device geometry. In order to determine the geometry of the device in vivo , this paper presents work on developing an auto-calibration procedure to measure the geometry of the robot using force sensors to move injector. The presented auto-calibration routine is able to identify the shape of the device to within 0.5 mm and 0.9°.

This paper presents the design and actuation of a six-degree-of-freedom (6-DOF) manipulator for a handheld instrument, known as "Micron," which performs active tremor compensation during microsurgery. The design incorporates a Gough-Stewart platform based on piezoelectric linear motor, with a specified minimum workspace of a cylinder 4 mm long and 4 mm in diameter at the end-effector. Given the stall force of the motors and the loading typically encountered in vitreoretinal microsurgery, the dimensions of the manipulator are optimized to tolerate a transverse load of 0.2 N on a remote center of motion near the midpoint of the tool shaft. The optimization yields a base diameter of 23 mm and a height of 37 mm. The fully handheld instrument includes a custom-built optical tracking system for control feedback, and an ergonomic housing to serve as a handle. The manipulation performance was investigated in both clamped and handheld conditions. In positioning experiments with varying side loads, the manipulator tolerates side load up to 0.25 N while tracking a sinusoidal target trajectory with less than 20 μm error. Physiological hand tremor is reduced by about 90% in a pointing task, and error less than 25 μm is achieved in handheld circle-tracing.

Prior work has highlighted the challenges faced by people with athetosis when trying to acquire on-screen targets using a mouse or trackball. The difficulty of positioning the mouse cursor within a confined area has been identified as a challenging task. We have developed a target acquisition assistance algorithm that features transition assistance via directional gain variation based on target prediction, settling assistance via gain reduction in the vicinity of a predicted target, and expansion of the predicted target as the cursor approaches it. We evaluated the algorithm on improving target acquisition efficiency among seven participants with athetoid cerebral palsy. Our results showed that the algorithm significantly reduced the overall movement time by about 20%. Considering the target acquisition occurs countless times in the course of regular computer use, the accumulative effect of such improvements can be significant for improving the efficiency of computer interaction among people with athetosis.

Micron is a fully handheld active micromanipulator that helps to improve position accuracy and precision in microsurgery by cancelling hand tremor. This work describes adaptation, tuning, and testing of the Micron system for stapedotomy, a microsurgical procedure performed in the middle ear to restore hearing that requires accurate manipulation in narrow spaces. Two end-effectors, a handle, and a brace (or rest) were designed and prototyped. The control system was adapted for the new hardware. The system was tested ex vivo in stapedotomy procedure comparing manually-performed and Micron-assisted surgical tasks. Tremor amplitude was found to be reduced significantly. Further testing is needed in order to obtain statistically significant results regarding other parameters dealing with regularity of the fenestra shape.

Gene therapies have emerged as a promising treatment for congestive heart failure, yet they lack a method for minimally invasive, uniform delivery. To address this need we developed Cerberus, a minimally invasive parallel wire robot for cardiac interventions. Prior work on Cerberus was limited to controlling the device using only position feedback. In order to ensure safety for both the patient and the device, as well as to improve the performance of the device, this paper presents work on enhancing the existing system with force feedback capabilities. By modeling the statics of the system and developing a tension distribution optimization technique, existing position control schemes were modified to a hybrid force/position controller. Experimental results show that using a hybrid force-position control scheme as opposed to position decreases positioning error by 38%.

This paper presents a technique for automated intraocular laser surgery using a handheld micromanipulator known as Micron. The novel handheld manipulator enables the automated scanning of a laser probe within a cylinder of 4 mm long and 4 mm in diameter. For the automation, the surface of the retina is reconstructed using a stereomicroscope, and then preplanned targets are placed on the surface. The laser probe is precisely located on the target via visual servoing of the aiming beam, while maintaining a specific distance above the surface. In addition, the system is capable of tracking the surface of the eye in order to compensate for any eye movement introduced during the operation. We compared the performance of the automated scanning using various control thresholds, in order to find the most effective threshold in terms of accuracy and speed. Given the selected threshold, we conducted the handheld operation above a fixed target surface. The average error and execution time are reduced by 63.6% and 28.5%, respectively, compared to the unaided trials. Finally, the automated laser photocoagulation was demonstrated also in an eye phantom, including compensation for the eye movement.

Vitreoretinal microsurgery requires precise hand-eye coordination to manipulate delicate structures within the eye on the order of tens of microns. To achieve these tasks, surgeons use tools of diameter 0.9 mm or less to access the eye's interior structures. The level of force required during these manipulations is often below the human tactile threshold, requiring the surgeon to rely on subtle visual cues or to apply larger forces above the tactile threshold for feedback. However, both of these methods can lead to tissue damage. Excursions can be made into tissues which are not felt by the surgeon, while larger forces have a higher chance of damaging tissue within the eye. To prevent damage to the retina and other anatomy, we present the implementation of hybrid position/force control operating in the sub-tactile force range for a handheld robotic system. This approach resulted in a 42% reduction in the mean force and 52% reduction in maximum force during peeling tasks.

This paper describes the design and preliminary testing of a planar parallel wire robot that adheres to the surface of the beating heart and provides a stable platform for minimally invasive epicardial therapies. The device is deployed through a small subxiphoid skin incision and attaches to the heart using suction. This methodology obviates mechanical stabilization and lung deflation, which are typically required during minimally invasive beating-heart surgery. The prototype design involves three vacuum chambers connected by two flexible arms. The chambers adhere to the epicardium, forming the vertices of a triangular base structure. Three cables connect a movable end-effector head to the three bases; the cables then pass out of the body to external actuators. The surgical tool moves within the triangular workspace to perform injections, ablation, or other tasks on the beating heart. Tests in vitro and in vivo were conducted to demonstrate the capabilities of the system. Tests in vivo successfully demonstrated the ability to deploy through a subxiphoid incision, adhere to the surface of the beating heart, move the surgical tool head within the robot's workspace, and perform injections into the myocardium.

Deep needle insertion into brain is important for both diagnostic and therapeutic clinical interventions. We have developed an automated system for robotically steering flexible needles within the brain to improve targeting accuracy. In this work, we have developed a finite element needle-tissue interaction model that allows for the investigation of safe parameters for needle steering. The tissue model implemented contains both hyperelastic and viscoelastic properties to simulate the instantaneous and time-dependent responses of brain tissue. Several needle models were developed with varying parameters to study the effects of the parameters on tissue stress, strain and strain rate during needle insertion and rotation. The parameters varied include needle radius, bevel angle, bevel tip fillet radius, insertion speed, and rotation speed. The results will guide the design of safe needle tips and control systems for intracerebral needle steering.

This paper presents a framework for localizing a miniature epicardial crawling robot, HeartLander, on the beating heart using only 6-degree-of-freedom position measurements from an electromagnetic position tracker and a dynamic surface model of the heart. Using only this information, motion and observation models of the system are developed such that a particle filter can accurately estimate not only the location of the robot on the surface of the heart, but also the pose of the heart in the world coordinate frame as well as the current physiological phase of the heart. The presented framework is then demonstrated in simulation on a dynamic 3-D model of the human heart and a robot motion model which accurately mimics the behavior of the HeartLander robot.

Performing micromanipulation and delicate operations in submillimeter workspaces is difficult because of destabilizing tremor and imprecise targeting. Accurate micromanipulation is especially important for microsurgical procedures, such as vitreoretinal surgery, to maximize successful outcomes and minimize collateral damage. Robotic aid combined with filtering techniques that suppress tremor frequency bands increases performance; however, if knowledge of the operator's goals is available, virtual fixtures have been shown to further improve performance. In this paper, we derive a virtual fixture framework for active handheld micromanipulators that is based on high-bandwidth position measurements rather than forces applied to a robot handle. For applicability in surgical environments, the fixtures are generated in real-time from microscope video during the procedure. Additionally, we develop motion scaling behavior around virtual fixtures as a simple and direct extension to the proposed framework. We demonstrate that virtual fixtures significantly outperform tremor cancellation algorithms on a set of synthetic tracing tasks (p < 0.05). In more medically relevant experiments of vein tracing and membrane peeling in eye phantoms, virtual fixtures can significantly reduce both positioning error and forces applied to tissue (p < 0.05).

This paper presents the first experimental results from human users of a new 6-degree-of-freedom handheld micromanipulator. This is the latest prototype of a fully-handheld system, known as "Micron," which performs active compensation of hand tremor for microsurgery. The manipulator is a miniature Gough-Stewart platform incorporating linear ultrasonic motors that provide a cylindrical workspace 4 mm long and 4 mm wide. In addition, the platform allows the possibility of imposing a remote center of motion for controlling motion not only at the tip but also at the entry point in the sclera of the eye. We demonstrate hand tremor reduction in both static and dynamic micromanipulation tasks on a rubber pad. The handheld performance is also evaluated in an artificial eye model while imposing a remote center of motion. In all cases, hand tremor is significantly reduced.

An active handheld micromanipulator has been developed to cancel hand tremor during microsurgery. The micromanipulator is also applicable in optical coherence tomography to improve the quality of scanning and minimize surgical risks during the scans. The manipulator can maneuver the tool tip with six degrees of freedom within a cylindrical workspace 4 mm in diameter and 4 mm high. The imaging system is equipped with a 25-gauge Fourier-domain common-path OCT probe. This paper introduces the handheld OCT imaging system and techniques involved and presents stabilized OCT images of A-mode and M-mode scans in air and live rabbit eyes. We show the first demonstration of OCT imaging using the active handheld micromanipulator in vivo.

Bevel-tipped flexible needles can be robotically steered to reach clinical targets along curvilinear paths in 3D. Manual needle insertion allows the clinician to control the insertion speed, ensuring patient safety. This paper presents a control law for automatic 3D steering of manually inserted flexible needles, enabling path-following control. A look-ahead proportional controller for position and orientation is presented. The look-ahead distance is a linear function of insertion speed. Simulations in a 3D brain-like environment demonstrate the performance of the proposed controller. Experimental results also show the feasibility of this technique in 2D and 3D environments.

Computer-aided intraocular surgery requires precise, real-time knowledge of the vasculature during retinal procedures such as laser photocoagulation or vessel cannulation. Because vitreoretinal surgeons manipulate retinal structures on the back of the eye through ports in the sclera, voluntary and involuntary tool motion rotates the eye in the socket and causes movement to the microscope view of the retina. The dynamic nature of the surgical workspace during intraocular surgery makes mapping, tracking, and localizing vasculature in real time a challenge. We present an approach that both maps and localizes retinal vessels by temporally fusing and registering individual-frame vessel detections. On video of porcine and human retina, we demonstrate real-time performance, rapid convergence, and robustness to variable illumination and tool occlusion.

This paper presents the design and analysis of a handheld manipulator for vitreoretinal microsurgery and other biomedical applications. The design involves a parallel micromanipulator utilizing six piezoelectric linear actuators, combining compactness with a large range of motion and relatively high stiffness. Given the available force of the actuators, the overall dimension of the micromanipulator was optimized considering realistic external loads on a remote center of motion representing the point of expected contact with the sclera of the eye during microsurgery. Based on optimization and workspace analysis, a benchtop version of the micromanipulator was built with a base diameter of 25 mm and a height of 50 mm. It provides a hemispherical workspace of 4.0 mm diameter at the tool tip. The manipulation performance of the constructed manipulator was measured under a lateral load applied at the remote center of motion. The micromanipulator tolerated side loads up to 200 mN.

Peeling delicate retinal membranes, which are often less than 5 µm thick, is one of the most challenging retinal surgeries. Preventing rips and tears caused by tremor and excessive force can decrease injury and reduce the need for follow up surgeries. We propose the use of a fully handheld microsurgical robot to suppress tremor while enforcing helpful constraints on the motion of the tool. Using stereo vision and tracking algorithms, the robot activates motion-scaled behavior as the tip reaches the surface, providing finer control during the critical step of engaging the membrane edge. A hard virtual fixture just below the surface limits the total downward force that can be applied. Furthermore, velocity limiting during the peeling helps the surgeon maintain a smooth, constant force while lifting and delaminating the membrane. On a phantom consisting of plastic wrap stretched across a rubber slide, we demonstrate our approach reduces maximum force by 40-70%.

Injecting clot-busting drugs such as t-PA into tiny vessels thinner than a human hair in the eye is a challenging procedure, especially since the vessels lie directly on top of the delicate and easily damaged retina. Various robotic aids have been proposed with the goal of increasing safety by removing tremor and increasing precision with motion scaling. We have developed a fully handheld micromanipulator, Micron, that has demonstrated reduced tremor when cannulating porcine retinal veins in an "open sky" scenario. In this paper, we present work towards handheld robotic cannulation with the goal of vision-based virtual fixtures guiding the tip of the cannula to the vessel. Using a realistic eyeball phantom, we address sclerotomy constraints, eye movement, and non-planar retina. Preliminary results indicate a handheld micromanipulator aided by visual control is a promising solution to retinal vessel occlusion.

HeartLander is a miniature mobile robot which adheres to and crawls over the surface of the beating heart to provide therapies in a minimally invasive manner. Although HeartLander inherently provides a stable operating platform, the motion of the surface of the heart remains an important factor in the operation of the robot. The quasi-periodic motion of the heart due to physiological cycles, respiration and the heartbeat, affects the ability of the robot to move, as well as localize accurately. In order to improve locomotion efficiency, as well as register different locations on the heart in physiological phase, two methods of identifying physiological phases are presented: sliding-window-based and model-based. In the sliding-window-based approach a vector of previous measurements is compared to previously learned motion templates to determine the current physiological phases, while the model-based approach learns a Fourier series model of the motion, and uses this model to estimate the current physiological phases using an Extended Kalman Filter (EKF). The two methods, while differing in approach, produce similarly accurate results on data recorded from animal experiments in vivo .

Highly accurate positioning is fundamental to the performance of vitreoretinal microsurgery. Of vitreoretinal procedures, membrane peeling is among the most prone to complications since extremely delicate manipulation of retinal tissue is required. Associated tool-to-tissue interaction forces are usually below the threshold of human perception, and the surgical tools are moved very slowly, within the 0.1-0.5 mm/s range. During the procedure, unintentional tool motion and excessive forces can easily give rise to vision loss or irreversible damage to the retina. A successful surgery includes two key features: controlled tremor-free tool motion and control of applied force. In this study, we present the potential benefits of a micro-force sensing robot in vitreoretinal surgery. Our main contribution is implementing fiber Bragg grating based force sensing in an active tremor canceling handheld micromanipulator, known as Micron, to measure tool-to-tissue interaction forces in real time. Implemented auditory sensory substitution assists in reducing and limiting forces. In order to test the functionality and performance, the force sensing Micron was evaluated in peeling experiments with adhesive bandages and with the inner shell membrane from chicken eggs. Our findings show that the combination of active tremor canceling together with auditory sensory substitution is the most promising aid that keeps peeling forces below 7 mN with a significant reduction in 2-20 Hz oscillations.

Gyroscope-free inertial measurement units have been gaining popularity in applications such as motion sensing of hand-held microsurgical instruments. Various accelerometer placement configurations have been proposed in the past. However, the effect of non-identical transfer functions of accelerometers on the accuracy of angular motion sensing has been underestimated and never been taken into consideration. In this paper, significant effect of different phases at different accelerometer outputs due to non-identical transfer functions on the accuracy is highlighted, and a method is proposed to make the transfer functions of all the accelerometers identical. Effectiveness of the method is confirmed with an experiment using ADXL-203 accelerometers.

Background: HeartLander is a miniature mobile robot designed to navigate over the epicardium of the beating heart for minimally invasive therapy. This paper presents a technique to decrease slippage and improve locomotion efficiency by synchronizing the locomotion with the intrapericardial pressure variations of the respiration and heartbeat cycles., Methods: Respiratory and heartbeat phases were detected in real time using a chest-mounted accelerometer during locomotion in a porcine model in vivo. Trials were conducted over the lateral aspect of the heart surface to test synchronized locomotion against an unsynchronized control., Results: Offline evaluation showed that the respiration and heartbeat algorithms had accuracies of 100% and 88%, respectively. Synchronized trials exhibited significantly lower friction, higher efficiency, and greater total distance traveled than control trials., Conclusion: Synchronization of the locomotion of HeartLander with respiration and heartbeat is feasible and results in safer and more efficient travel on the beating heart., (Copyright © 2011 John Wiley & Sons, Ltd.)