Search

Background: Virtual reality (VR) technology is emerging as a powerful tool in medical training and has potential benefits for paramedic education. Aim: The aim of this paper is to report the development of ParaVR, which uses VR to maintain paramedics' skills. Methods: Computer scientists at the University of Chester and the Welsh Ambulance Services NHS Trust (WAST) developed ParaVR in four stages: identifying requirements and specifications; alpha version development; beta version development; and management—development of software, further funding and commercialisation. Results: Needle cricothyrotomy and needle thoracostomy emerged as candidates for the prototype ParaVR. The Oculus Rift head-mounted display was combined with Novint Falcon haptic device and a virtual environment crafted using 3D modelling software, which was ported to the Oculus Go virtual reality headset and the Google Cardboard VR platform. Conclusion: VR is an emerging educational tool with the potential to enhance paramedic skills development and maintenance. The ParaVR programme is the first step in the authors' development, testing and scaling up of this technology.

Surgical smoke removal algorithms can improve the quality of intra-operative imaging and reduce hazards in image-guided surgery, a highly desirable post-process for many clinical applications. These algorithms also enable effective computer vision tasks for future robotic surgery. In this article, we present a new unsupervised learning framework for high-quality pixel-wise smoke detection and removal. One of the well recognized grand challenges in using convolutional neural networks (CNNs) for medical image processing is to obtain intra-operative medical imaging datasets for network training and validation, but availability and quality of these datasets are scarce. Our novel training framework does not require ground-truth image pairs. Instead, it learns purely from computer-generated simulation images. This approach opens up new avenues and bridges a substantial gap between conventional non-learning based methods and which requiring prior knowledge gained from extensive training datasets. Inspired by the Generative Adversarial Network (GAN), we have developed a novel generative-collaborative learning scheme that decomposes the de-smoke process into two separate tasks: smoke detection and smoke removal. The detection network is used as prior knowledge, and also as a loss function to maximize its support for training of the smoke removal network. Quantitative and qualitative studies show that the proposed training framework outperforms the state-of-the-art de-smoking approaches including the latest GAN framework (such as PIX2PIX). Although trained on synthetic images, experimental results on clinical images have proved the effectiveness of the proposed network for detecting and removing surgical smoke on both simulated and real-world laparoscopic images.

Navigating a powered wheelchair and avoiding collisions is often a daunting task for new wheelchair users. It takes time and practice to gain the coordination needed to become a competent driver and this can be even more of a challenge for someone with a disability. We present a cost-effective virtual reality (VR) application that takes advantage of consumer level VR hardware. The system can be easily deployed in an assessment centre or for home use, and does not depend on a specialized high-end virtual environment such as a Powerwall or CAVE. This paper reviews previous work that has used virtual environments technology for training tasks, particularly wheelchair simulation. We then describe the implementation of our own system and the first validation study carried out using thirty three able bodied volunteers. The study results indicate that at a significance level of 5 percent then there is an improvement in driving skills from the use of our VR system. We thus have the potential to develop the competency of a wheelchair user whilst avoiding the risks inherent to training in the real world. However, the occurrence of cybersickness is a particular problem in this application that will need to be addressed.

The requirement for training surgical procedures without exposing the patient to additional risk is well accepted and is part of a national drive in the UK and internationally. Computer-based simulations are important in this context, including neurosurgical resident training. The objective of this study is to evaluate the effectiveness of a custom-built virtual environment in assisting training of a ventriculostomy procedure. The training tool (called VCath) has been developed as an app for a tablet platform to provide easy access and availability to trainees. The study was conducted at the first boot camp organized for all year-one trainees in neurosurgery in the UK. The attendees were randomly distributed between the VCath training group and the control group. Efficacy of performing ventriculostomy for both groups was assessed at the beginning and end of the study using a simulated insertion task. Statistically significant changes in performance of selecting the burr hole entry point, the trajectory length and duration metrics for the VCath group, together with a good indicator of improved normalized jerk (representing the speed and smoothness of arm motion), all suggest that there has been a higher-level cognitive benefit to using VCath. The app is successful as it is focused on the cognitive task of ventriculostomy, encouraging the trainee to rehearse the entry point and use anatomical landmarks to create a trajectory to the target. In straight-line trajectory procedures such as ventriculostomy, cognitive task-based education is a useful adjunct to traditional methods and may reduce the learning curve and ultimately improve patient safety.

This paper presents recent additions to our Wheelchair-VR application, in particular the use of different drive configurations. We have previously shown that Wheelchair-VR can be used to improve driving skills. Here we consider the utility of the application in allowing users who are in the process of purchasing or upgrading a wheelchair to experience different configurations and options in a cost-effective virtual environment. A preliminary study is presented, which suggests that this approach can be effective.

PurposeWe review augmented (AR) and virtual reality (VR) applications in radiotherapy as found in the scientific literature and highlight future developments enabled by the use of small mass-produced devices and portability of techniques developed in other fields to radiotherapy.AnalysisThe application of AR and VR within radiotherapy is still in its infancy, with the notable exception of training and teaching applications. The relatively high cost of equipment needed to generate a realistic 3D effect seems one factor that has slowed down its use, but also the sheer amount of image data is relatively recent, were radiotherapy professionals are only beginning to explore how to use this to its full potential. This increased availability of 3D data in radiotherapy will drive the application of AR and VR in radiotherapy to efficiently recognise and extract key features in the data to act on in clinical decision making.ConclusionThe development of small mass-produced tablet devices coming on the market will allow the user to interact with computer-generated information more easily, facilitating the application of AR and VR. The increased connectivity enabling virtual presence of remote multidisciplinary team meetings heralds significant changes to how radiotherapy professionals will work, to the benefit of our patients.

International audience; Purpose Training in Interventional Radiology currently uses the apprenticeship model, where clinical and technical skills of invasive procedures are learnt during practice in patients. This apprenticeship training method is increasingly limited by regulatory restrictions on working hours, concerns over patient risk through trainees' inexperience and the variable exposure to case mix and emergencies during training. To address this, we have developed a computer-based simulation of visceral needle puncture procedures. Methods A real-time framework has been built that includes: segmentation, physically based modelling, haptics rendering, pseudo-ultrasound generation and the concept of a physical mannequin. It is the result of a close collaboration between different universities, involving computer scientists, clinicians, clinical engineers and occupational psychologists. Results The technical implementation of the framework is a robust and real-time simulation environment combining a physical platform and an immersive computerized virtual environment. The face, content and construct validation have been previously assessed, showing the reliability and effectiveness of this framework, as well as its potential for teaching visceral needle puncture. Conclusion A simulator for ultrasound-guided liver biopsy has been developed. It includes functionalities and metrics extracted from cognitive task analysis. This framework can be useful during training, particularly given the known difficulties in gaining significant practice of core skills in patients.

This review paper discusses the role of haptics within virtual medical training applications, particularly, where it can be used to aid a practitioner to learn and practice a task. The review summarizes aspects to be considered in the deployment of haptics technologies in medical training. First, both force/torque and tactile feedback hardware solutions that are currently produced commercially and in academia are reviewed, followed by the available haptics-related software and then an in-depth analysis of medical training simulations that include haptic feedback. The review is summarized with scrutiny of emerging technologies and discusses future directions in the field.

The use of Virtual Environments has been widely reported as a method of teaching anatomy. Generally such environments only convey the shape of the anatomy to the student. We present the Bangor Augmented Reality Education Tool for Anatomy (BARETA), a system that combines Augmented Reality (AR) technology with models produced using Rapid Prototyping (RP) technology, to provide the student with stimulation for touch as well as sight. The principal aims of this work were to provide an interface more intuitive than a mouse and keyboard, and to evaluate such a system as a viable supplement to traditional cadaver based education.

Commercial interventional radiology vascular simulators emulate instrument navigation and device deployment, though none supports the Seldinger technique, which provides initial access to the vascular tree. This paper presents a novel virtual environment for teaching this core skill. Our simulator combines two haptic devices: vessel puncture with a virtual needle and catheter and guidewire manipulation. The simulation software displays the instrument interactions with the vessels. Instruments are modelled using a mass-spring approximation, while efficient collision detection and collision response allow real time interactions. Experienced interventional radiologists evaluated the haptic components of our simulator as realistic and accurate. The vessel puncture haptic device proposes a first prototype to simulate the Seldinger technique. Our simulator presents realistic instrument behaviour when compared to real instruments in a vascular phantom. This paper presents the first simulator to train the Seldinger technique. The preliminary results confirm its utility for interventional radiology training.

This paper discusses the design issues and implementation details of building a medical training simulator. Example projects that have been undertaken by the Visualization and Medical Graphics group at Bangor University and our collaborators are used to illustrate the points made. A detailed case study is then presented of a virtual environment designed to train the Seldinger Technique, a common procedure in interventional radiology. The paper will introduce a medical practitioner to the technology behind a medical virtual environment. It will also provide an engineer with an overview of many of the issues that need to be considered when undertaking to build such an application. The paper ends with the author’s views on future developments in this exciting domain.

The medical domain provides excellent opportunities for communication and teaching of healthcare issues using computer graphics, visualization techniques, and virtual environments. Possible applications include anatomical educational tools; patient education; diagnostic aids; virtual autopsies; planning and guidance aids; skills training; and computer augmented reality. Both clinicians and patients can benefit from the appropriate use of tools that make use of these technologies. This paper provides an overview of the state-of-the-art technologies in this exciting field, including detailed examples from our research. The term cybermedicine is discussed and issues for effective cybermedicine are highlighted.

If we were to have a Grid infrastructure for visualization, what technologies would be needed to build such an infrastructure, what kind of applications would benefit from it, and what challenges are we facing in order to accomplish this goal? In this survey paper, we make use of the term ‘visual supercomputing’ to encapsulate a subject domain concerning the infrastructural technology for visualization. We consider a broad range of scientific and technological advances in computer graphics and visualization, which are relevant to visual supercomputing. We identify the state-of-the-art technologies that have prepared us for building such an infrastructure. We examine a collection of applications that would benefit enormously from such an infrastructure, and discuss their technical requirements. We propose a set of challenges that may guide our strategic efforts in the coming years.

We present a simulator for guided needle puncture procedures. Our aim is to provide an effective training tool for students in interventional radiology (IR) using actual patient data and force feedback within an immersive virtual environment (VE). Training of the visual and motor skills required in IR is an apprenticeship which still consists of close supervision using the model: (i) see one, (ii) do one, and (iii) teach one. Training in patients not only has discomfort associated with it, but provides limited access to training scenarios, and makes it difficult to train in a time efficient manner. Currently, the majority of commercial products implementing a medical VE still focus on laparoscopy where eye–hand coordination and sensation are key issues. IR procedures, however, are far more reliant on the sense of touch. Needle guidance using ultrasound or computed tomography (CT) images is also widely used. Both of these are areas that have not been fully addressed by other medical VEs. This paper provides details of how we are developing an effective needle guidance simulator. The project is a multi-disciplinary collaboration involving practising interventional radiologists and computer scientists.

Op3D is a new visualisation system for a surgeon. For the first time, patient-specific medical data sets can be rendered in three dimensions using a powerful compute server, with the reconstructions being delivered across the computer network in real time to the operating theatre. A laptop serves as the client computer and an easy-to-use interface has been developed for the surgeon to interact with and interrogate the patient data. This paper describes the Op3D system and presents the initial results from its use in hepatopancreatic surgery. Examples are given from both preoperative and intraoperative use.

Applications of virtual reality (VR) and augmented reality (AR) in medicine require real-time visualization and modeling of large three-dimensional data sets. Consequently, these applications require powerful computation, extensive high-bandwidth memory, and fast communication links. In the past, the manufacturers of medical imaging equipment produced their own special-purpose proprietary hardware for image processing and solid graphics. Due to the developments in computer hardware in general and in graphics accelerators in particular, there is a trend toward replacing the proprietary hardware off-the-shelf (OTS) equipment. Computer graphics itself has advanced in its quest for realism. Generic algorithms such as shading, texture mapping, and volume rendering have been developed to meet the resultant ever increasing requirements. Advances in both the OTS CPU and graphics hardware have enabled real-time implementations of these algorithms, thereby facilitating many of the medical VR/AR applications used today. The development of graphics libraries such as OpenGL has also been an important factor. These libraries provide an underlying portable software platform that optimizes the utilization of the available graphics hardware. OpenGL has become a standard graphics application programming interface, particularly for graphics-intensive applications, and more and more OTS systems provide hardware implementations of OpenGL commands. The review paper follows the evolution of these technologies and examines their crucial role in enabling the appearance of the current VR/AR applications in medicine and provides a look at current trends and future possibilities.

We have designed a configurable virtual environment to train rugby ball passing skills. Seeking to validate the system’s ability to correctly aid training, two experiments were performed. Ten participants took part in ball passing activities, which were used to compare the combinations of different user positions relative to the physical screen, the use of stereoscopic presentation and the use of a floor screen to extend the field of view of the virtual scene. Conversely to what was expected, the results indicate that the participants did not respond well to simulated target distances, and only the users physical distance from the screen had an effect on the distance thrown.

We have designed and developed a virtual environment to train rugby ball passing skills. Seeking to validate the system's ability to correctly aid training, an initial experiment was performed to examine the effect of stereoscopic technology and the physical screen's setup on the user's ability to perceive virtual distances correctly. Four participants with at least 10 years' experience of playing local-level rugby took part in the experiment to compare the combinations of different user positions compared to the physical screen, the use of stereoscopic presentation and the use of a floor screen to extend the field of view of the virtual scene. Conversely to what was expected, the participants did not respond well to perceived distance and these variables had little effect on the distance thrown.

Two different approaches for the preparation of novel cost-effective molecular haptic applications (Figure 1) are described. The former utilises Perl scripting within a commercial molecular modelling package to generate static / animated H3D scene graphs for haptic CPK space-filling atomic perception. Within the second approach, key chemical concepts such as reactivity, aciditity and periodicity can be investigated using prototype H3D user interfaces.

Haptics force-feedback technology is fast becoming a consumer product and no longer only found in research laboratories. The emergence of the budget Falcon device (Novint Technologies, Inc., USA) represents a key step in the dissemination of haptics technology as it offers this functionality to home users, in particular to games players. Haptics has the potential to revolutionise the Human Computer Interface if novel and creative software solutions can be found to utilise it. Currently developing for haptics requires low level programming knowledge, which is often a barrier to uptake. This paper looks at how haptics support can be integrated into an X3D authored virtual world using an open source haptics library via the Scene Authoring Interface (SAI). We supply a partial implementation of a Java wrapping to the HAPI open-source haptics library and provide a demonstration of its use within the Xj3D browser through SAI. This work is intended to contribute to a possible future haptics extension of the ISO X3D standard.

Interventional Radiology is a rapidly expanding speciality using minimally invasive techniques to treat a multitude of clinical problems. Current work in progress aims to create an affordable virtual training tool to reduce training times and patient risk during a trainee practitioners learning cycle. The procedure of arterial catheterisation has been broken down into a number of subtasks, one of which requires an operator to locate the femoral artery pulse by palpation. This is performed in preparation for a needle insertion to allow the entry of a guide wire and catheter into the patient. This paper presents the current state of research into a unique solution for affordable haptic simulation of pulse palpation in a virtual environment.

The Department of Chemistry and the School of Informatics at the University of Wales, Bangor are working together to create tools for the visualization of molecular quantum dynamics. This paper presents the results of our initial work. A prototype Molecular Visualiser (MV) application has been developed based on Web3D standards, plus extensions for support of haptic interaction. MV provides the user with visualizations of molecular systems, potential energy surfaces, and wavepacket dynamics. These can be displayed in a web browser using VRML, or be delivered to a virtual environment in which haptic properties have been assigned based on the molecular dynamics of the system. The use of MV for both research and teaching is discussed.

This paper describes an innovative system called Op3D that delivers high performance visualization to a hospital operating theatre. This is the first time that a hepato-pancreatic surgeon has been able to benefit from 3D (three-dimensional) visualization tools at the sharp end of his clinical practice. Early results are supporting our hypothesis that interactive access to 3D visualization and reconstruction from the operating theatre improves the prospect of successful resection of hepato-pancreatic tumors (and could easily be exploited in other surgical areas too). The implementation of the system is discussed in detail, including all of the software and hardware components involved. The paper ends with a discussion of the future potential of the system to take advantage of high performance computing, visual area networking, and the grid.

In this paper, the application of the Virtual Reality Modeling Language (VRML) for efficient database visualization is analyzed. With the help of JAVA programming, three examples of automatic visualization from a database containing 3-D Geometry are given. The first example is used to create basic geometries. The second example is used to create cylinders with a defined start point and end point. The third example is used to processs data from an old copper mine complex in Cheshire, United Kingdom. Interactive 3-D visualization of all geometric data in an online database is achieved with JSP technology.

The NOVICE project (Network-Oriented Visualization in a Clinical Environment) is developing a system for medical image data analysis and visualization through the Web with the intention of demonstrating the effectiveness of such a system in a clinical environment. One element being addressed by NOVICE is the provision of teleradiology tools. The paper describes how three-dimensional models can be included in a teleradiology application by utilising the Virtual Reality Modelling Language (VRML) and multi-user extensions, including annotation tools. The result is the world's first Web based system for 3D teleradiology. We present a detailed overview of the implementation and operation of this system, and indicate results from early use.

This paper describes methods of online translation of text data files into a visually more intuitive 3-D format. Relatively simple text files are used as examples here, but the approach described can easily be extended to more complicated tasks. Java Servlet Pages (JSP) dynamic homepage technology is used to provide online access and data processing. Different processing methods are used by the example applications.

This chapter provides an introduction to the technology behind medical virtual reality (VRe). The six main VRe technology components required are described: visual displays; tracking; input devices; haptics; audio; and computer hardware. Example uses in medical applications are provided. An overview of the medical areas where VRe is already being applied is then given, including training, diagnosis, planning and computer-augmented surgery.

Information concerning the application of proton beams in radiotherapy of ocular tumours is provided using conventional website technology by a number of treatment facilities around the world. We hypothesized, however, that many of the key concepts would be better conveyed by an interactive computer model utilizing virtual reality technology. We describe the implementation, using Virtual Reality Modelling Language (VRML), of such a model, the Proton Therapy Concepts Demonstrator (PTCD). The World-Wide-Web-accessible PTCD is intended to provide information useful both to trainee and to qualified clinical staff and also to patients. A model was created of the radiotherapy room that was linked to an interactive model depicting a simple explanation of the process of proton eye radiotherapy. This model also allows the user to explore specific elements of the treatment planning and delivery process, such as beam collimation and range modulation. A further level of detail has been provided by a dynamic model demonstrating how a specific modulated dose distribution is achieved as the time integration of discrete modulated Bragg peaks. We believe this VR-based model has the potential of enabling users to gain more rapid insight into the proton therapy process. Its development as an educational tool is continuing, following feedback collected from clinical staff and patients.

jgViz uses standard grid technologies and Chromium cluster graphics software to schedule the best available distributed graphics pipeline. Visualization of large data sets using computer graphics techniques has become a mainstream requirement for many scientists and engineers. However, achieving interactive performance when rendering a complex visualization often requires high-performance computing facilities. One solution is to exploit real-time graphics accelerators, which are used in the full range of computing devices from games consoles and set-top boxes to supercomputers and advanced training simulators. At the same time, the service infrastructure for distributed networks has produced the third generation of the connected world: the grid. jgViz, our Java-implemented grid visualization system, uses grid functionality to enable transparent access to reliable parallel graphics pipelines. Lightweight and highly portable, jgViz aims to be plug-and-play for the existing grid resource allocation mechanism