Search

Navigating a powered wheelchair and avoiding collisions is often a daunting task for new wheelchair users. It takes time and practice to gain the coordination needed to become a competent driver and this can be even more of a challenge for someone with a disability. We present a cost-effective virtual reality (VR) application that takes advantage of consumer level VR hardware. The system can be easily deployed in an assessment centre or for home use, and does not depend on a specialized high-end virtual environment such as a Powerwall or CAVE. This paper reviews previous work that has used virtual environments technology for training tasks, particularly wheelchair simulation. We then describe the implementation of our own system and the first validation study carried out using thirty three able bodied volunteers. The study results indicate that at a significance level of 5 percent then there is an improvement in driving skills from the use of our VR system. We thus have the potential to develop the competency of a wheelchair user whilst avoiding the risks inherent to training in the real world. However, the occurrence of cybersickness is a particular problem in this application that will need to be addressed.

Background and objective: While Minimally Invasive Surgery (MIS) offers considerable benefits to patients, it also imposes big challenges on a surgeon’s performance due to well-known issues and restrictions associated with the field of view (FOV), hand-eye misalignment and disorientation, as well as the lack of stereoscopic depth perception in monocular endoscopy. Augmented Reality (AR) technology can help to overcome these limitations by augmenting the real scene with annotations, labels, tumour measurements or even a 3D reconstruction of anatomy structures at the target surgical locations. However, previous research attempts of using AR technology in monocular MIS surgical scenes have been mainly focused on the information overlay without addressing correct spatial calibrations, which could lead to incorrect localization of annotations and labels, and inaccurate depth cues and tumour measurements. In this paper, we present a novel intra-operative dense surface reconstruction framework that is capable of providing geometry information from only monocular MIS videos for geometry-aware AR applications such as site measurements and depth cues. We address a number of compelling issues in augmenting a scene for a monocular MIS environment, such as drifting and inaccurate planar mapping. Methods: A state-of-the-art Simultaneous Localization And Mapping (SLAM) algorithm used in robotics has been extended to deal with monocular MIS surgical scenes for reliable endoscopic camera tracking and salient point mapping. A robust global 3D surface reconstruction framework has been developed for building a dense surface using only unorganized sparse point clouds extracted from the SLAM. The 3D surface reconstruction framework employs the Moving Least Squares (MLS) smoothing algorithm and the Poisson surface reconstruction framework for real time processing of the point clouds data set. Finally, the 3D geometric information of the surgical scene allows better understanding and accurate placement AR augmentations based on a robust 3D calibration. Results: We demonstrate the clinical relevance of our proposed system through two examples: (a) measurement of the surface; (b) depth cues in monocular endoscopy. The performance and accuracy evaluations of the proposed framework consist of two steps. First, we have created a computer-generated endoscopy simulation video to quantify the accuracy of the camera tracking by comparing the results of the video camera tracking with the recorded ground-truth camera trajectories. The accuracy of the surface reconstruction is assessed by evaluating the Root Mean Square Distance (RMSD) of surface vertices of the reconstructed mesh with that of the ground truth 3D models. An error of 1.24 mm for the camera trajectories has been obtained and the RMSD for surface reconstruction is 2.54 mm, which compare favourably with previous approaches. Second, in vivo laparoscopic videos are used to examine the quality of accurate AR based annotation and measurement, and the creation of depth cues. These results show the potential promise of our geometry-aware AR technology to be used in MIS surgical scenes. Conclusions: The results show that the new framework is robust and accurate in dealing with challenging situations such as the rapid endoscopy camera movements in monocular MIS scenes. Both camera tracking and surface reconstruction based on a sparse point cloud are effective and operated in real-time. This demonstrates the potential of our algorithm for accurate AR localization and depth augmentation with geometric cues and correct surface measurements in MIS with monocular endoscopes.

In vascular interventional radiology, procedures generally start with the Seldinger technique to access the vasculature, using a needle through which a guidewire is inserted, followed by navigation of catheters within the vessels. Visual and tactile skills are learnt in a patient apprenticeship which is expensive and risky for patients. We propose a training alternative through a new virtual simulator supporting the Seldinger technique: ImaGiNe (imaging guided interventional needle) Seldinger. It is composed of two workstations: (1) a simulated pulse is palpated, in an immersive environment, to guide needle puncture and (2) two haptic devices provide a novel interface where a needle can direct a guidewire and catheter within the vessel lumen, using virtual fluoroscopy. Different complexities are provided by 28 real patient datasets. The feel of the simulation is enhanced by replicating, with the haptics, real force and flexibility measurements. A preliminary validation study has demonstrated training effectiveness for skills transfer.

International audience; Purpose Training in Interventional Radiology currently uses the apprenticeship model, where clinical and technical skills of invasive procedures are learnt during practice in patients. This apprenticeship training method is increasingly limited by regulatory restrictions on working hours, concerns over patient risk through trainees' inexperience and the variable exposure to case mix and emergencies during training. To address this, we have developed a computer-based simulation of visceral needle puncture procedures. Methods A real-time framework has been built that includes: segmentation, physically based modelling, haptics rendering, pseudo-ultrasound generation and the concept of a physical mannequin. It is the result of a close collaboration between different universities, involving computer scientists, clinicians, clinical engineers and occupational psychologists. Results The technical implementation of the framework is a robust and real-time simulation environment combining a physical platform and an immersive computerized virtual environment. The face, content and construct validation have been previously assessed, showing the reliability and effectiveness of this framework, as well as its potential for teaching visceral needle puncture. Conclusion A simulator for ultrasound-guided liver biopsy has been developed. It includes functionalities and metrics extracted from cognitive task analysis. This framework can be useful during training, particularly given the known difficulties in gaining significant practice of core skills in patients.

The use of Virtual Environments has been widely reported as a method of teaching anatomy. Generally such environments only convey the shape of the anatomy to the student. We present the Bangor Augmented Reality Education Tool for Anatomy (BARETA), a system that combines Augmented Reality (AR) technology with models produced using Rapid Prototyping (RP) technology, to provide the student with stimulation for touch as well as sight. The principal aims of this work were to provide an interface more intuitive than a mouse and keyboard, and to evaluate such a system as a viable supplement to traditional cadaver based education.

Commercial interventional radiology vascular simulators emulate instrument navigation and device deployment, though none supports the Seldinger technique, which provides initial access to the vascular tree. This paper presents a novel virtual environment for teaching this core skill. Our simulator combines two haptic devices: vessel puncture with a virtual needle and catheter and guidewire manipulation. The simulation software displays the instrument interactions with the vessels. Instruments are modelled using a mass-spring approximation, while efficient collision detection and collision response allow real time interactions. Experienced interventional radiologists evaluated the haptic components of our simulator as realistic and accurate. The vessel puncture haptic device proposes a first prototype to simulate the Seldinger technique. Our simulator presents realistic instrument behaviour when compared to real instruments in a vascular phantom. This paper presents the first simulator to train the Seldinger technique. The preliminary results confirm its utility for interventional radiology training.

The medical domain provides excellent opportunities for communication and teaching of healthcare issues using computer graphics, visualization techniques, and virtual environments. Possible applications include anatomical educational tools; patient education; diagnostic aids; virtual autopsies; planning and guidance aids; skills training; and computer augmented reality. Both clinicians and patients can benefit from the appropriate use of tools that make use of these technologies. This paper provides an overview of the state-of-the-art technologies in this exciting field, including detailed examples from our research. The term cybermedicine is discussed and issues for effective cybermedicine are highlighted.

This paper provides an overview of the main techniques being used for three-dimensional (3D) visualization of medical data sets and highlights some of the clinical benefits that can be obtained. One of the major advantages of using a 3D representation is that all of the slice data produced by the latest multislice CT and high gradient MR scanners can be utilized, and then presented to the clinician in an intuitive format. Continued advances in technology mean that high resolution 3D representations of patient specific anatomy can now be routinely obtained and so provide valuable input to diagnosis, planning and navigation tasks. Examples from these areas are presented and illustrated below. Future developments and possibilities are also discussed.

The Op3D visualization system allows, for the first time, a surgeon in the operating theatre to interrogate patient-specific medical data sets rendered in three dimensions using high-performance computing. The hypothesis of this research is that the success rate of hepato-pancreatic surgical resections can be improved by replacing the light box with an interactive 3D representation of the medical data in the operating theatre.A laptop serves as the client computer and an easy-to-use interface has been developed for the surgeon to interact with and interrogate the patient data. To date, 16 patients have had 3D reconstructions of their DICOM data sets, including preoperative interrogation and planning of surgery.Interrogation of the 3D images live in theatre and comparison with the surgeons' operative findings (including intraoperative ultrasound) led to the operation being abandoned in 25% of cases, adoption of an alternative surgical approach in 25% of cases, and helpful image guidance for successful resection in 50% of cases.The clinical value of the latest generation of scanners and digital imaging techniques cannot be realized unless appropriate dissemination of the images takes place. This project has succeeded in translating the image technology into a user-friendly form and delivers 3D reconstructions of patient-specific data to the "sharp end"-the surgeon undertaking the tumor resection in theatre, in a manner that allows interaction and interpretation. More time interrogating the 3D data sets preoperatively would help reduce the incidence of abandoned operations-this is part of the surgeons' learning curve. We have developed one of the first practical applications to benefit from remote visualization, and certainly the first medical visualization application of this kind.

Two-dimensional asymmetry, border irregularity, colour variegation and diameter (ABCD) features are important indicators currently used for computer-assisted diagnosis of malignant melanoma (MM); however, they often prove to be insufficient to make a convincing diagnosis. Previous work has demonstrated that 3D skin surface normal features in the form of tilt and slant pattern disruptions are promising new features independent from the existing 2D ABCD features. This work investigates that whether improved lesion classification can be achieved by combining the 3D features with the 2D ABCD features. Experiments using a nonlinear support vector machine classifier show that many combinations of the 2D ABCD features and the 3D features can give substantially better classification accuracy than using (1) single features and (2) many combinations of the 2D ABCD features. The best 2D and 3D feature combination includes the overall 3D skin surface disruption, the asymmetry and all the three colour channel features. It gives an overall 87.8 % successful classification, which is better than the best single feature with 78.0 % and the best 2D feature combination with 83.1 %. These demonstrate that (1) the 3D features have additive values to improve the existing lesion classification and (2) combining the 3D feature with all the 2D features does not lead to the best lesion classification. The two ABCD features not selected by the best 2D and 3D combination, namely (1) the border feature and (2) the diameter feature, were also studied in separate experiments. It found that inclusion of either feature in the 2D and 3D combination can successfully classify 3 out of 4 lesion groups. The only one group not accurately classified by either feature can be classified satisfactorily by the other. In both cases, they have shown better classification performances than those without the 3D feature in the combinations. This further demonstrates that (1) the 3D feature can be used to improve the existing 2D-based diagnosis and (2) including the 3D feature with subsets of the 2D features can be used in distinguishing different benign lesion classes from MM. It is envisaged that classification performance may be further improved if different 2D and 3D feature subsets demonstrated in this study are used in different stages to target different benign lesion classes in future studies.

A proposed method visualizes the surface appearance of living human brain tissue. The goal is to investigate whether realistic models of living anatomy are possible and, if so, whether they provide added value to anatomy education and training simulators. From calibrated photography of exposed brain tissue and suitable alternatives, experiments provided data for a bidirectional reflectance distribution function, which was then used for rendering. Employing a GPU, real-time visualization of the brain's surface supported ambient occlusion, advanced texturing, subsurface scattering, and specularity.

We have investigated the ability to quantitate atherosclerosis in the aortic arch of the Watanabe rabbit using noninvasive 3-D ultrasound. Our methodology utilizes postprocessing of videotaped freehand 2-D interrogations to form a compound 3-D data block. Structures may then be segmented on the attributed grey-scale level and volumes measured. Analysis of 3-D reconstructions revealed a low echo structure in the aortic arch of atherosclerotic rabbits, absent in nonatherosclerotic rabbits, at recognized sites of plaque predilection. This structure volume correlated closely with fatty streak volume determined from histology (r = 0.890). During a 30-week study, this structure volume increased in untreated animals, but was blocked by treatment with the antiatherosclerotic agent probucol. Thus, a new 3-D ultrasound methodology has been used noninvasively to detect and quantitate a low echo structure corresponding to fatty streaks in the Watanabe rabbit aortic arch. This new methodology could potentially aid plaque burden quantification in human peripheral arteries.

Medical simulators provide a risk-free environment for trainee doctors to practice and improve their skills. UltraPulse is a new tactile system designed to utilise focussed airborne ultrasound to mimic a pulsation effect such as that of a human arterial pulse. In this paper, we focus on the construction of the haptics component, which can later be integrated into a variety of medical procedure training simulators.

VCath is a neurosurgery training tool for the catheterization of the lateral ventricle that has been designed for use on tablet devices. We believe this is the first use of a tablet (iPad) for this purpose and demonstrates future utility for this approach, particularly for Objective Structured Clinical Exams (OSCEs). This paper outlines the implementation and use of VCath.

This paper introduces a novel technique for the visualization of blood (or other fluid) flowing through a complex 3D network of vessels. The Directed Particle System (DPS) approach is loosely based on the computer graphics concept of flocking agents. It has been developed and optimised to provide effective real time visualization and qualitative simulation of fluid flow. There are many potential applications of DPS, and one example - a decision support tool for coronary collateralization - is discussed.

Osteoarthritis of the hip is commonly caused by the repetitive contact between abnormal skeletal prominences between the anterosuperior femoral head-neck junction and the rim of the acetabular socket. Current methods for estimating femoroacetabular impingement by analyzing the sphericity of the femoral head require manual measurements which are both inaccurate and open to interpretation. In this research we provide a prototype software tool for improving this estimation.

Interventional Radiology procedures (e.g., angioplasty, embolization, stent graft placement) provide minimally invasive therapy to treat a wide range of conditions. These procedures involve the use of flexible tipped guidewires to advance diagnostic or therapeutic catheters into a patient's vascular or visceral anatomy. This paper presents a real-time physically based hybrid modeling approach to simulating guidewire insertions. The long, slender body of the guidewire shaft is simulated using nonlinear elastic Cosserat rods, and the shorter flexible tip composed of a straight, curved, or angled design is modeled using a more efficient generalized bending model. Therefore, the proposed approach efficiently computes intrinsic dynamic behaviors of guidewire interactions within vascular structures. The efficacy of the proposed method is demonstrated using detailed numerical simulations inside 3-D blood vessel structures derived from preprocedural volumetric data. A validation study compares positions of four physical guidewires deployed within a vascular phantom, with the co-ordinates of the corresponding simulated guidewires within a virtual model of the phantom. An optimization algorithm is also implemented to further improve the accuracy of the simulation. The presented simulation model is suitable for interactive virtual reality-based training and for treatment planning.

We propose a method to automatically tune a patient-based virtual environment training simulator for abdominal needle insertion. The key attributes to be customized in our framework are the elasticity of soft-tissues and the respiratory model parameters. The estimation is based on two 3D Computed Tomography (CT) scans of the same patient at two different time steps. Results are presented on four patients and show that our new method leads to better results than our previous studies with manually tuned parameters.

Objectives Mentoring, for physicians and surgeons in training, is advocated as an essential adjunct in work-based learning, providing support in career and non-career related issues. The World Wide Web (WWW) has evolved, as a technology, to become more interactive and person centric, tailoring itself to the individual needs of the user. This changing technology may open new avenues to foster mentoring in medicine. Design, Systematic Review, Main Outcome Measures A search of the MEDLINE database from 1950 to 2012 using the PubMed interface, combined with manual cross-referencing was performed using the following strategy: (“mentors”[MeSH Terms] OR “mentors”[All Fields] OR “mentor”[All Fields]) AND (“internet”[MeSH Terms] OR “internet”[All Fields]) AND (“medicine”[MeSH Terms] OR “medicine”[All Fields]) AND (“humans”[MeSH Terms] AND English[lang]). Abstracts were screened for relevance (UJ) to the topic; eligibility for inclusion was simply on screening for relevance to online mentoring and web-based technologies. Results Forty-five papers were found, of which 16 were relevant. All studies were observational in nature. To date, all medical mentoring applications utilizing the World Wide Web have enjoyed some success limited by Web 1.0 and 2.0 technologies. Conclusions With the evolution of the WWW through 1.0, 2.0 and 3.0 generations, the potential for meaningful tele- and distance mentoring has greatly improved. Some engagement has been made with these technological advancements, however further work is required to fully realize the potential of these technologies.

A SensAble Omni force feedback device has been modified to increase the face validity of a needle insertion simulation. The new end effector uses a real needle hub and shortened needle shaft in place of the Omni's pre-fitted pen shaped end effector. This modification facilitates correct procedural training through the simulation of co-located visual and haptic cues in an augmented reality approach to simulation. The development of the new end effector is described and a pictorial guide to its manufacture and the fitting process is provided. Initial results from face validation studies bode well for the fidelity of this low cost device.

In this paper we summarize the progress of the Web3D scene graph model, and associated standards, specifically Extensible 3D (X3D) in the domain of medical simulation. Historically, the Web3D nodesets have focused on the representation and rendering of point, line or surface geometry. More recently, significant progress in X3D Volume rendering has been made available through the co-operative DICOM work item, n-Dimensional Presentation States. However, here we outline the need for a standard for simulation meshes and review several related approaches. As a result, we propose preliminary requirements for a simulation mesh standard and provide several use case scenarios of how Web3D and haptic technologies can aid the fulfillment of these requirements. We conclude with an X3D proposal to describe simulation meshes for soft (deformable) bodies.

This paper presents an advanced method of visualizing the surface appearance of living brain tissue. We have been granted access to the operating theatre during neurosurgical procedures to obtain colour data via calibrated photography of exposed brain tissue. The specular reflectivity of the brain's surface is approximated by analyzing a gelatine layer applied to animal flesh. This provides data for a bidirectional reflectance distribution function (BRDF) that is then used the rendering process. Rendering is achieved in realtime by utilizing the GPU, and includes support for ambient occlusion, advanced texturing, sub surface scattering and specularity. Our goal is to investigate whether realistic visualizations of living anatomy can be produced and so provide added value to anatomy education.

This paper reports on a low cost system for training ultrasound imaging techniques. The need for such training is particularly acute in developing countries where typically ultrasound scanners remain idle due to the lack of experienced sonographers. The system described below is aimed at a PC platform but uses interface components from the Nintendo Wii games console. The training software is being designed to support a variety of patient case studies, and also supports remote tutoring over the internet.

The original publication is available at www.springerlink.com; International audience; Purpose : We present here a simulator for interventional radiology focusing on percutaneous transhepatic cholangiography (PTC). This procedure consists of inserting a needle into the biliary tree using fluoroscopy for guidance. Methods : The requirements of the simulator have been driven by a task analysis. The three main components have been identified: the respiration, the real-time X-ray display (fluoroscopy) and the haptic rendering (sense of touch). The framework for modelling the respiratory motion is based on kinematics laws and on the Chainmail algorithm. The fluoroscopic simulation is performed on the graphic card and makes use of the Beer-Lambert law to compute the X-ray attenuation. Finally, the haptic rendering is integrated to the virtual environment and takes into account the soft-tissue reaction force feedback and maintenance of the initial direction of the needle during the insertion. Results : Five training scenarios have been created using patient-specific data. Each of these provides the user with variable breathing behaviour, fluoroscopic display tuneable to any device parameters and needle force feedback. Conclusions : A detailed task analysis has been used to design and build the PTC simulator described in this paper. The simulator includes real-time respiratory motion with two independent parameters (rib kinematics and diaphragm action), on-line fluoroscopy implemented on the Graphics Processing Unit and haptic feedback to feel the soft-tissue behaviour of the organs during the needle insertion.

As the power of personal computing increases, there is a greater demand for high fidelity virtual environments for medical training simulators. However, it is still a challenge to provide realistic soft tissue deformation especially when haptic feedback is also required. This paper presents a new approach to soft tissue deformation using a novel Charged Particle method to efficiently model both the structure and haptic properties of anatomy in real time.

Interventional Radiology (IR) procedures are minimally invasive, targeted treatments performed using imaging for guidance. Needle puncture using ultrasound, x-ray, or computed tomography (CT) images is a core task in the radiology curriculum, and we are currently developing a training simulator for this. One requirement is to include support for physically-based simulation of x-ray images from CT data sets. In this paper, we demonstrate how to exploit the capability of today's graphics cards to efficiently achieve this on the Graphics Processing Unit (GPU) and compare performance with an efficient software only implementation using the Central Processing Unit (CPU).

The ever improving price-performance ratio of personal computers is making a significant contribution to widening the accessibility of training simulators for a wide variety of medical procedures. High fidelity solutions are becoming more and more sought after. However, the problem of providing realistic soft tissue deformation in real time remains, particularly if haptic interaction is also required. This paper presents a new approach for efficient soft tissue deformation using particle systems to model both structure and haptic properties of anatomy. We are applying this technique to a simulator for interventional radiology procedures, but it can easily be adapted for other medical domains.

Transcranial Magnetic Stimulation (TMS) is the process in which electrical activity in the brain is influenced by a pulsed magnetic field. Common practice is to align an electromagnetic coil with points of interest identified on the surface of the brain, from an MRI scan of the subject. The coil can be tracked using optical sensors, enabling the targeting information to be calculated and displayed on a local workstation. In this paper we explore the hypothesis that using an Augmented Reality (AR) interface for TMS will improve the efficiency of carrying out the procedure. We also aim to provide a flexible infrastructure that if required, can seamlessly deploy processing power from a remote high performance computing resource.

The use of stereoscopy to enhance the immersive experience obtained from a virtual environment is well known, and many medical applications can benefit from this technology. The stereoscopic projection sessions at the last two MMVR conferences contain many excellent examples. Stereoscopy need not be expensive to implement, and can easily be provided on a desktop PC. This paper looks at the different options available today, from active stereo, to passive stereo, to autostereoscopic displays, and their deployment on both high-end and low-end systems. The potential benefits of these different approaches are illustrated with examples from medical visualisation projects currently being carried out at the Manchester Visualization Centre (MVC).

Mastoidectomy is one of the most common surgical procedures relating to the petrous bone. In this paper we describe our preliminary results in the realization of a virtual reality mastoidectomy simulator. Our system is designed to work on patient-specific volumetric object models directly derived from 3D CT and MRI images. The paper summarizes the detailed task analysis performed in order to define the system requirements, introduces the architecture of the prototype simulator, and discusses the initial feedback received from selected end users.

Information concerning the application of proton beams in radiotherapy of ocular tumours is provided using conventional website technology by a number of treatment facilities around the world. We hypothesized, however, that many of the key concepts would be better conveyed by an interactive computer model utilizing virtual reality technology. We describe the implementation, using Virtual Reality Modelling Language (VRML), of such a model, the Proton Therapy Concepts Demonstrator (PTCD). The World-Wide-Web-accessible PTCD is intended to provide information useful both to trainee and to qualified clinical staff and also to patients. A model was created of the radiotherapy room that was linked to an interactive model depicting a simple explanation of the process of proton eye radiotherapy. This model also allows the user to explore specific elements of the treatment planning and delivery process, such as beam collimation and range modulation. A further level of detail has been provided by a dynamic model demonstrating how a specific modulated dose distribution is achieved as the time integration of discrete modulated Bragg peaks. We believe this VR-based model has the potential of enabling users to gain more rapid insight into the proton therapy process. Its development as an educational tool is continuing, following feedback collected from clinical staff and patients.

We have used new developments in computer technology and the Internet to create a small program that simulates catheterization of the lateral ventricle. The program can run on most personal computers connected to the Internet. The program allows trainee surgeons to practice the technique with varying degrees of visual feedback--and no risk to the patient. It allows them to learn both the technique and the associated anatomy. The trainees can be assessed while performing the procedure. This is a small, early-stage application of virtual reality in surgical education. There is a demand for surgical training techniques that expose the patient to no risk; the use of computers is appropriate to meet this demand. The technique presented here requires further development; in particular, it needs a standardized assessment element that will allow it to be tested by established surgeons and trainees. The real test of a good training technique is how well it discriminates between trainee and trainer.

A system is described that rapidly produces a regular 3-dimensional (3-D) data block suitable for processing by conventional image analysis and volume measurement software. The system uses electromagnetic spatial location of 2-dimensional (2-D) freehand-scanned ultrasound B-mode images, custom-built signal-conditioning hardware, UNIX-based computer processing and an efficient 3-D reconstruction algorithm. Utilisation of images from multiple angles of insonation, "compounding," reduces speckle contrast, improves structure coherence within the reconstructed grey-scale image and enhances the ability to detect structure boundaries and to segment and quantify features. Volume measurements using a series of water-filled latex and cylindrical foam rubber phantoms with volumes down to 0.7 mL show that a high degree of accuracy, precision and reproducibility can be obtained. Extension of the technique to handle in vivo data sets by allowing physiological criteria to be taken into account in selecting the images used for construction is also illustrated.