Search

The UK average survival rate from out of hospital cardiac arrest (OHCA) is 8.6%, which is significantly lower than in comparable countries where survival rates can exceed 20%. A cardiac arrest victim is two to four times more likely to survive OHCA with bystander cardiopulmonary resuscitation (CPR). Mandatory teaching of CPR in schools is an effective way, endorsed by the World Health Organization, to train the entire population and improve the bystander CPR rate. Despite this, as with other UK home nations, there is significant variation in provision of CPR training within schools in Wales. Virtual reality (VR) technology offers an accessible, immersive way to teach CPR skills to schoolchildren. Computer scientists at the University of Chester and the Welsh Ambulance Services NHS Trust developed Virtual Cardio Pulmonary Resuscitation (VCPR), which can be used to teach children CPR skills. There were three stages: identifying requirements and specifications; development of a prototype; and management—development of software, further funding and exploring opportunities for commercialisation.

Background: Virtual reality (VR) technology is emerging as a powerful tool in medical training and has potential benefits for paramedic education. Aim: The aim of this paper is to report the development of ParaVR, which uses VR to maintain paramedics' skills. Methods: Computer scientists at the University of Chester and the Welsh Ambulance Services NHS Trust (WAST) developed ParaVR in four stages: identifying requirements and specifications; alpha version development; beta version development; and management—development of software, further funding and commercialisation. Results: Needle cricothyrotomy and needle thoracostomy emerged as candidates for the prototype ParaVR. The Oculus Rift head-mounted display was combined with Novint Falcon haptic device and a virtual environment crafted using 3D modelling software, which was ported to the Oculus Go virtual reality headset and the Google Cardboard VR platform. Conclusion: VR is an emerging educational tool with the potential to enhance paramedic skills development and maintenance. The ParaVR programme is the first step in the authors' development, testing and scaling up of this technology.

Surgical smoke removal algorithms can improve the quality of intra-operative imaging and reduce hazards in image-guided surgery, a highly desirable post-process for many clinical applications. These algorithms also enable effective computer vision tasks for future robotic surgery. In this article, we present a new unsupervised learning framework for high-quality pixel-wise smoke detection and removal. One of the well recognized grand challenges in using convolutional neural networks (CNNs) for medical image processing is to obtain intra-operative medical imaging datasets for network training and validation, but availability and quality of these datasets are scarce. Our novel training framework does not require ground-truth image pairs. Instead, it learns purely from computer-generated simulation images. This approach opens up new avenues and bridges a substantial gap between conventional non-learning based methods and which requiring prior knowledge gained from extensive training datasets. Inspired by the Generative Adversarial Network (GAN), we have developed a novel generative-collaborative learning scheme that decomposes the de-smoke process into two separate tasks: smoke detection and smoke removal. The detection network is used as prior knowledge, and also as a loss function to maximize its support for training of the smoke removal network. Quantitative and qualitative studies show that the proposed training framework outperforms the state-of-the-art de-smoking approaches including the latest GAN framework (such as PIX2PIX). Although trained on synthetic images, experimental results on clinical images have proved the effectiveness of the proposed network for detecting and removing surgical smoke on both simulated and real-world laparoscopic images.

Virtual reality entertainment and serious games popularity has continued to rise but the processes for level design for VR games has not been adequately researched. Our paper contributes LevelEd VR; a generic runtime virtual reality level editor that supports the level design workflow used by developers and can potentially support user generated content. We evaluated our LevelEd VR application and compared it to an existing workflow of Unity on a desktop. Our current research indicates that users are accepting of such a system, and it has the potential to be preferred over existing workflows for VR level design. We found that the primary benefit of our system is an improved sense of scale and perspective when creating the geometry and implementing gameplay. The paper also contributes some best practices and lessons learned from creating a complex virtual reality tool, such as LevelEd VR.

We present VRIA, a Web-based framework for creating Immersive Analytics (IA) experiences in Virtual Reality. VRIA is built upon WebVR, A-Frame, React and D3.js, and offers a visualization creation workflow which enables users, of different levels of expertise, to rapidly develop Immersive Analytics experiences for the Web. The use of these open-standards Web-based technologies allows us to implement VR experiences in a browser and offers strong synergies with popular visualization libraries, through the HTML Document Object Model (DOM). This makes VRIA ubiquitous and platform-independent. Moreover, by using WebVR's progressive enhancement, the experiences VRIA creates are accessible on a plethora of devices. We elaborate on our motivation for focusing on open-standards Web technologies, present the VRIA creation workflow and detail the underlying mechanics of our framework. We also report on techniques and optimizations necessary for implementing Immersive Analytics experiences on the Web, discuss scalability implications of our framework, and present a series of use case applications to demonstrate the various features of VRIA. Finally, we discuss current limitations of our framework, the lessons learned from its development, and outline further extensions.

As the healthcare enterprise is adopting novel imaging and health-assessment technologies, we are facing unprecedented requirements in information sharing, patient empowerment, and care coordination within the system. Medical experts not only within US, but around the world should be empowered through collaboration capabilities on 3D data to enable solutions for complex medical problems that will save lives. The fast-growing number of 3D medical ‘images' and their derivative information must be shared across the healthcare enterprise among stakeholders with vastly different perspectives and different needs. The demand for 3D data visualization is driving the need for increased accessibility and sharing of 3D medical image presentations, including their annotations and their animations. As patients have to make decisions about their health, empowering them with the right tools to understand a medical procedure is essential both in the decision-making process and for knowledge sharing.

Navigating a powered wheelchair and avoiding collisions is often a daunting task for new wheelchair users. It takes time and practice to gain the coordination needed to become a competent driver and this can be even more of a challenge for someone with a disability. We present a cost-effective virtual reality (VR) application that takes advantage of consumer level VR hardware. The system can be easily deployed in an assessment centre or for home use, and does not depend on a specialized high-end virtual environment such as a Powerwall or CAVE. This paper reviews previous work that has used virtual environments technology for training tasks, particularly wheelchair simulation. We then describe the implementation of our own system and the first validation study carried out using thirty three able bodied volunteers. The study results indicate that at a significance level of 5 percent then there is an improvement in driving skills from the use of our VR system. We thus have the potential to develop the competency of a wheelchair user whilst avoiding the risks inherent to training in the real world. However, the occurrence of cybersickness is a particular problem in this application that will need to be addressed.

The potential of augmented reality (AR) technology to assist minimally invasive surgery (MIS) lies in its computational performance and accuracy in dealing with challenging MIS scenes. Even with the latest hardware and software technologies, achieving both real-time and accurate augmented information overlay in MIS is still a formidable task. In this Letter, the authors present a novel real-time AR framework for MIS that achieves interactive geometric aware AR in endoscopic surgery with stereo views. The authors’ framework tracks the movement of the endoscopic camera and simultaneously reconstructs a dense geometric mesh of the MIS scene. The movement of the camera is predicted by minimising the re-projection error to achieve a fast tracking performance, while the three-dimensional mesh is incrementally built by a dense zero mean normalised cross-correlation stereo-matching method to improve the accuracy of the surface reconstruction. The proposed system does not require any prior template or pre-operative scan and can infer the geometric information intra-operatively in real time. With the geometric information available, the proposed AR framework is able to interactively add annotations, localisation of tumours and vessels, and measurement labelling with greater precision and accuracy compared with the state-of-the-art approaches.

This research project developed a Virtual Reality (VR) training simulator for the CPR procedure. This is designed for use training school children. It can also form part of a larger system for training paramedics with VR. The simulator incorporates a number of advanced VR technologies including Oculus Rift and Leap motion. We have gained input from NHS paramedics and several related organisation to design the system and provide feedback and evaluation of the preliminary working prototype.

This research project developed a Virtual Reality (VR) training simulator for paramedic procedures. Currently needle cricothyroidotomy and chest drain are modelled, which could form part of a larger system for training paramedics with VR in various other procedures. The simulator incorporates a number of advanced VR technologies including Oculus Rift and haptic feedback. We have gained input and feedback from NHS paramedics and several related organisation to design the system and provide feedback and evaluation of the preliminary working prototype.

We describe a mixed reality environment that has been designed as an aid for training driving skills for a powered wheelchair. Our motivation is to provide an improvement on a previous virtual reality wheelchair driving simulator, with a particular aim to remove any cybersickness effects. The results of a validation test are presented that involved 35 able bodied volunteers divided into three groups: mixed reality trained, virtual reality trained, and a control group. No significant differences in improvement was found between the groups but there is a notable trend that both the mixed reality and virtual reality groups improved more than the control group. Whereas the virtual reality group experienced discomfort (as measured using a simulator sickness questionnaire), the mixed reality group experienced no side effects.

We present ongoing work to develop a virtual reality environment for the cognitive rehabilitation of patients as a part of their recovery from a stroke. A stroke causes damage to the brain and problem solving, memory and task sequencing are commonly affected. The brain can recover to some extent, however, and stroke patients have to relearn to carry out activities of daily learning. We have created an application called VIRTUE to enable such activities to be practiced using immersive virtual reality. Gamification techniques enhance the motivation of patients such as by making the level of difficulty of a task increase over time. The design and implementation of VIRTUE is presented together with the results of a small acceptability study.

We developed an application that makes indoor modelling accessible by utilizing consumer grade technology in the form of Apple's ARKit and a smartphone to assist with serious games level design. We compared our system to that of a tape measure and a system based on an infra-red depth sensor and application. We evaluated the accuracy and efficiency of each system over four different measuring tasks of increasing complexity. Our results suggest that our application is more accurate than the depth sensor system and as accurate and more time efficient as the tape measure over several tasks. Participants also showed a preference to our LevelEd AR application over the depth sensor system regarding usability.

We report on the design, implementation and evaluation of , a framework for building immersive analytics (IA) solutions in Web-based Virtual Reality (VR), built upon WebVR, A-Frame, React and D3. The recent emergence of affordable VR interfaces have reignited the interest of researchers and developers in exploring new, immersive ways to visualize data. In particular, the use of open-standards web-based technologies for implementing VR in a browser facilitates the ubiquitous and platform-independent adoption of IA systems. Moreover, such technologies work in synergy with established visualization libraries, through the HTML document object model (DOM). We discuss high-level features of and present a preliminary user experience evaluation of one of our use cases.

The requirement for training surgical procedures without exposing the patient to additional risk is well accepted and is part of a national drive in the UK and internationally. Computer-based simulations are important in this context, including neurosurgical resident training. The objective of this study is to evaluate the effectiveness of a custom-built virtual environment in assisting training of a ventriculostomy procedure. The training tool (called VCath) has been developed as an app for a tablet platform to provide easy access and availability to trainees. The study was conducted at the first boot camp organized for all year-one trainees in neurosurgery in the UK. The attendees were randomly distributed between the VCath training group and the control group. Efficacy of performing ventriculostomy for both groups was assessed at the beginning and end of the study using a simulated insertion task. Statistically significant changes in performance of selecting the burr hole entry point, the trajectory length and duration metrics for the VCath group, together with a good indicator of improved normalized jerk (representing the speed and smoothness of arm motion), all suggest that there has been a higher-level cognitive benefit to using VCath. The app is successful as it is focused on the cognitive task of ventriculostomy, encouraging the trainee to rehearse the entry point and use anatomical landmarks to create a trajectory to the target. In straight-line trajectory procedures such as ventriculostomy, cognitive task-based education is a useful adjunct to traditional methods and may reduce the learning curve and ultimately improve patient safety.

Planning treatment of acute cardiac events that limit the blood supply to major organs is particularly difficult for interventional cardiologists. The treatment of pathologies, such as vascular stenosis, can have numerous unforeseen consequences as the blood resumes its flow. Their decisions are largely based on 2D medical imagery and their own experience. This work in progress presents a virtual reality blood simulation tool that will augment and improve a clinicians decision-making arsenal, and an outline of the technologies required for this component of the cyber hospital of the future. The tool displays and provides interaction with vital information on how blood may continue to flow through the cardiovascular system after treatment. The blood flow will be simulated using a bespoke implementation of the increasingly effective smoothed particle hydrodynamics model.

We have developed an augmented reality tool for radiotherapy to view the real world scene, i.e. the patient on a treatment couch, combined with computer graphics content, such as planning image data and any defined outlines of organ structures. We have deployed our software to a number of consumer electronics devices (iPad, Android tablets, MS HoloLens). We suggest that, in contrast to other augmented reality tools explored for radiotherapy [1], due to the wide availability and low cost of the hardware platforms considered, associated with their increasing computational and graphic power, our system has strong potential as a tool for visualization of medical information for clinicians and other radiotherapy professionals, as a device for patient positioning for radiotherapy treatment, and as an educational tool for patients to visualize their treatment and demonstrate to patients e.g. the importance of compliance with instructions around bladder filling and rectal suppositories. Accuracy of virtual content placement and user evaluation of our system has been experimentally investigated.

This paper presents recent additions to our Wheelchair-VR application, in particular the use of different drive configurations. We have previously shown that Wheelchair-VR can be used to improve driving skills. Here we consider the utility of the application in allowing users who are in the process of purchasing or upgrading a wheelchair to experience different configurations and options in a cost-effective virtual environment. A preliminary study is presented, which suggests that this approach can be effective.

Medical training systems often require the use of haptic interaction with deformable models and high visual fidelity. It is necessary to get a compromise between the geometric model and the deformation model in order to achieve the required frame rate for haptic interaction of 1000Hz. Frequently low level geometric models are used to describe the geometry of the object. This produces large data structures for which it is difficult to reach real-time force feedback.This paper proposes to use a novel model based on Bezier hyperpatches to represent the object, computing the deformation using a mass-spring system. This model can represent free form objects in a concise way. A small test application to simulate palpation with a haptic probe is presented.

Mixed Reality (MR) is of increasing interest within technology- driven modern medicine but is not yet used in everyday practice. This situation is changing rapidly, however, and this paper explores the emergence of MR technology and the importance of its utility within medical applications. A classification of medical MR has been obtained by applying an unbiased text mining method to a database of 1,403 relevant research papers published over the last two decades. The classification results reveal a taxonomy for the development of medical MR research during this period as well as suggesting future trends. We then use the classification to analyse the technology and applications developed in the last five years. Our objective is to aid researchers to focus on the areas where technology advancements in medical MR are most needed, as well as providing medical practitioners with a useful source of reference.

PurposeWe review augmented (AR) and virtual reality (VR) applications in radiotherapy as found in the scientific literature and highlight future developments enabled by the use of small mass-produced devices and portability of techniques developed in other fields to radiotherapy.AnalysisThe application of AR and VR within radiotherapy is still in its infancy, with the notable exception of training and teaching applications. The relatively high cost of equipment needed to generate a realistic 3D effect seems one factor that has slowed down its use, but also the sheer amount of image data is relatively recent, were radiotherapy professionals are only beginning to explore how to use this to its full potential. This increased availability of 3D data in radiotherapy will drive the application of AR and VR in radiotherapy to efficiently recognise and extract key features in the data to act on in clinical decision making.ConclusionThe development of small mass-produced tablet devices coming on the market will allow the user to interact with computer-generated information more easily, facilitating the application of AR and VR. The increased connectivity enabling virtual presence of remote multidisciplinary team meetings heralds significant changes to how radiotherapy professionals will work, to the benefit of our patients.

In vascular interventional radiology, procedures generally start with the Seldinger technique to access the vasculature, using a needle through which a guidewire is inserted, followed by navigation of catheters within the vessels. Visual and tactile skills are learnt in a patient apprenticeship which is expensive and risky for patients. We propose a training alternative through a new virtual simulator supporting the Seldinger technique: ImaGiNe (imaging guided interventional needle) Seldinger. It is composed of two workstations: (1) a simulated pulse is palpated, in an immersive environment, to guide needle puncture and (2) two haptic devices provide a novel interface where a needle can direct a guidewire and catheter within the vessel lumen, using virtual fluoroscopy. Different complexities are provided by 28 real patient datasets. The feel of the simulation is enhanced by replicating, with the haptics, real force and flexibility measurements. A preliminary validation study has demonstrated training effectiveness for skills transfer.

International audience; Purpose Training in Interventional Radiology currently uses the apprenticeship model, where clinical and technical skills of invasive procedures are learnt during practice in patients. This apprenticeship training method is increasingly limited by regulatory restrictions on working hours, concerns over patient risk through trainees' inexperience and the variable exposure to case mix and emergencies during training. To address this, we have developed a computer-based simulation of visceral needle puncture procedures. Methods A real-time framework has been built that includes: segmentation, physically based modelling, haptics rendering, pseudo-ultrasound generation and the concept of a physical mannequin. It is the result of a close collaboration between different universities, involving computer scientists, clinicians, clinical engineers and occupational psychologists. Results The technical implementation of the framework is a robust and real-time simulation environment combining a physical platform and an immersive computerized virtual environment. The face, content and construct validation have been previously assessed, showing the reliability and effectiveness of this framework, as well as its potential for teaching visceral needle puncture. Conclusion A simulator for ultrasound-guided liver biopsy has been developed. It includes functionalities and metrics extracted from cognitive task analysis. This framework can be useful during training, particularly given the known difficulties in gaining significant practice of core skills in patients.

Control of a powered wheelchair is often not intuitive, making training of new users a challenging and sometimes hazardous task. Collisions, due to a lack of experience can result in injury for the user and other individuals. By conducting training activities in virtual reality (VR), we can potentially improve driving skills whilst avoiding the risks inherent to the real world. However, until recently VR technology has been expensive and limited the commercial feasibility of a general training solution. We describe Wheelchair-Rift, a cost effective prototype simulator that makes use of the Oculus Rift head mounted display and the Leap Motion hand tracking device. It has been assessed for face validity by a panel of experts from a local Posture and Mobility Service. Initial results augur well for our cost-effective training solution.

This paper presents a virtual environment for training femoral palpation and needle insertion, the opening steps of many interventional radiology procedures. A novel augmented reality simulation called PalpSim has been developed that allows the trainees to feel a virtual patient using their own hands. The palpation step requires both force and tactile feedback. For the palpation haptics effect, two off-the-shelf force feedback devices have been linked together to provide a hybrid device that gives five degrees of force feedback. This is combined with a custom built hydraulic interface to provide a pulse like tactile effect. The needle interface is based on a modified PHANTOM Omni end effector that allows a real interventional radiology needle to be mounted and used during simulation. While using the virtual environment, the haptics hardware is masked from view using chroma-key techniques. The trainee sees a computer generated patient and needle, and interacts using their own hands. This simulation provides a high level of face validity and is one of the first medical simulation devices to integrate haptics with augmented reality.

Guidewire insertion is an imperative task of minimally invasive medical procedures. During the procedure, surgeons need to steer long flexible thin wires through patient's blood vessels to reach a clinical target. In this paper, we present a novel approach to model haptics of guidewire insertion process for training simulation. The algorithm also allows for the analysis of the insertion process through subtle physical behaviours of guidewires via force feedbacks. The method includes a 6-DoF dynamic coupling between a rigid body, i.e. the virtual tool and the deformation of the wire simulated as an elastic rod. Instead of using the frictional contact force or the acceleration of the guidewire tip for haptic feedbacks, we compute constrained forces by directly connecting the virtual tool to the end of the guidewire. Therefore, the coupling scheme transmits haptic interactions through constrained dynamics between the virtual tool and the guidewire. Both positional and rotational control modes are implemented and evaluated with respect to the dynamics of the guidewire, user inputs and feedback forces. Experiments highlight the usability of our algorithm for an insertion procedure simulation with complex blood vessel structures. Copyright © 2011 John Wiley & Sons, Ltd.

This review paper discusses the role of haptics within virtual medical training applications, particularly, where it can be used to aid a practitioner to learn and practice a task. The review summarizes aspects to be considered in the deployment of haptics technologies in medical training. First, both force/torque and tactile feedback hardware solutions that are currently produced commercially and in academia are reviewed, followed by the available haptics-related software and then an in-depth analysis of medical training simulations that include haptic feedback. The review is summarized with scrutiny of emerging technologies and discusses future directions in the field.

The use of Virtual Environments has been widely reported as a method of teaching anatomy. Generally such environments only convey the shape of the anatomy to the student. We present the Bangor Augmented Reality Education Tool for Anatomy (BARETA), a system that combines Augmented Reality (AR) technology with models produced using Rapid Prototyping (RP) technology, to provide the student with stimulation for touch as well as sight. The principal aims of this work were to provide an interface more intuitive than a mouse and keyboard, and to evaluate such a system as a viable supplement to traditional cadaver based education.

Commercial interventional radiology vascular simulators emulate instrument navigation and device deployment, though none supports the Seldinger technique, which provides initial access to the vascular tree. This paper presents a novel virtual environment for teaching this core skill. Our simulator combines two haptic devices: vessel puncture with a virtual needle and catheter and guidewire manipulation. The simulation software displays the instrument interactions with the vessels. Instruments are modelled using a mass-spring approximation, while efficient collision detection and collision response allow real time interactions. Experienced interventional radiologists evaluated the haptic components of our simulator as realistic and accurate. The vessel puncture haptic device proposes a first prototype to simulate the Seldinger technique. Our simulator presents realistic instrument behaviour when compared to real instruments in a vascular phantom. This paper presents the first simulator to train the Seldinger technique. The preliminary results confirm its utility for interventional radiology training.

This paper discusses the design issues and implementation details of building a medical training simulator. Example projects that have been undertaken by the Visualization and Medical Graphics group at Bangor University and our collaborators are used to illustrate the points made. A detailed case study is then presented of a virtual environment designed to train the Seldinger Technique, a common procedure in interventional radiology. The paper will introduce a medical practitioner to the technology behind a medical virtual environment. It will also provide an engineer with an overview of many of the issues that need to be considered when undertaking to build such an application. The paper ends with the author’s views on future developments in this exciting domain.

This paper provides a survey of medical applications that make use of Web3D technologies, covering the period from 1995 to 2005. We assess the impact that Web3D has made on medical education and training during this time and highlight current and future trends. The applications identified are categorized into: general education tools; tools for diagnosis; procedures training; and collaborative training. A summary of work that has been carried out to validate these tools is also included in the survey.

The medical domain provides excellent opportunities for communication and teaching of healthcare issues using computer graphics, visualization techniques, and virtual environments. Possible applications include anatomical educational tools; patient education; diagnostic aids; virtual autopsies; planning and guidance aids; skills training; and computer augmented reality. Both clinicians and patients can benefit from the appropriate use of tools that make use of these technologies. This paper provides an overview of the state-of-the-art technologies in this exciting field, including detailed examples from our research. The term cybermedicine is discussed and issues for effective cybermedicine are highlighted.

We have designed Li Tu (Laı'Tu - Light Tube) as a customisable and low-cost (ca 30 Euros) human-computer interface. It is composed of an acrylic tube, a ball-bearing mirror, six LEDs and a webcam. Touching the tube causes frustrated internal reflection of light due to a change in the critical angle at the acrylic-skin boundary. Scattered light within the tube is reflected off the mirror into the camera at the opposite end for image processing. Illuminated contact regions in the video frames are segmented and processed to generate 2D information such as: pitch and volume, or x and y coordinates of a graphic. We demonstrate the functionality of Li Tu both as a musical instrument and as an interactive computer graphics controller. For example, various musical notes can be generated by touching specific regions around the surface of the tube. Volume can be controlled by sliding a finger down the tube and pitch by sliding the finger radially. We demonstrate the adaptable nature of Li Tu's touch interface and discuss our plans to explore future physical modifications of the device.

We are developing a novel device for measuring hand power grip using frustrated total internal reflection of light in acrylic. Our method uses a force sensitive resistor to calibrate the force of a power grip as a function of the area and light intensity. This research is work in progress but results so far augur well for its applicability in medical and other application areas. The grip measurement device allows the patient and doctor to see the change in grip over time and projects this information directly onto the back of the patient's hand.

Work in progress for the development of a novel virtual training environment for training a kidney biopsy procedure is presented. Our goal is to provide an affordable high fidelity simulation through the integration of some of the latest off-the-shelf technology components. The range of forces that are encountered during this procedure have been recorded using a custom designed force sensitive glove and then applied within the simulation.

The medical domain provides excellent opportunities for the application of computer graphics, visualization and virtual environments, with the potential to help improve healthcare and bring benefits to patients. This survey paper provides a comprehensive overview of the state-of-the-art in this exciting field. It has been written from the perspective of both computer scientists and practising clinicians and documents past and current successes together with the challenges that lie ahead. The article begins with a description of the software algorithms and techniques that allow visualization of and interaction with medical data. Example applications from research projects and commercially available products are listed, including educational tools; diagnostic aids; virtual endoscopy; planning aids; guidance aids; skills training; computer augmented reality and use of high performance computing. The final section of the paper summarizes the current issues and looks ahead to future developments.

If we were to have a Grid infrastructure for visualization, what technologies would be needed to build such an infrastructure, what kind of applications would benefit from it, and what challenges are we facing in order to accomplish this goal? In this survey paper, we make use of the term ‘visual supercomputing’ to encapsulate a subject domain concerning the infrastructural technology for visualization. We consider a broad range of scientific and technological advances in computer graphics and visualization, which are relevant to visual supercomputing. We identify the state-of-the-art technologies that have prepared us for building such an infrastructure. We examine a collection of applications that would benefit enormously from such an infrastructure, and discuss their technical requirements. We propose a set of challenges that may guide our strategic efforts in the coming years.

We present a simulator for guided needle puncture procedures. Our aim is to provide an effective training tool for students in interventional radiology (IR) using actual patient data and force feedback within an immersive virtual environment (VE). Training of the visual and motor skills required in IR is an apprenticeship which still consists of close supervision using the model: (i) see one, (ii) do one, and (iii) teach one. Training in patients not only has discomfort associated with it, but provides limited access to training scenarios, and makes it difficult to train in a time efficient manner. Currently, the majority of commercial products implementing a medical VE still focus on laparoscopy where eye–hand coordination and sensation are key issues. IR procedures, however, are far more reliant on the sense of touch. Needle guidance using ultrasound or computed tomography (CT) images is also widely used. Both of these are areas that have not been fully addressed by other medical VEs. This paper provides details of how we are developing an effective needle guidance simulator. The project is a multi-disciplinary collaboration involving practising interventional radiologists and computer scientists.

This paper provides an overview of the main techniques being used for three-dimensional (3D) visualization of medical data sets and highlights some of the clinical benefits that can be obtained. One of the major advantages of using a 3D representation is that all of the slice data produced by the latest multislice CT and high gradient MR scanners can be utilized, and then presented to the clinician in an intuitive format. Continued advances in technology mean that high resolution 3D representations of patient specific anatomy can now be routinely obtained and so provide valuable input to diagnosis, planning and navigation tasks. Examples from these areas are presented and illustrated below. Future developments and possibilities are also discussed.

The Op3D visualization system allows, for the first time, a surgeon in the operating theatre to interrogate patient-specific medical data sets rendered in three dimensions using high-performance computing. The hypothesis of this research is that the success rate of hepato-pancreatic surgical resections can be improved by replacing the light box with an interactive 3D representation of the medical data in the operating theatre.A laptop serves as the client computer and an easy-to-use interface has been developed for the surgeon to interact with and interrogate the patient data. To date, 16 patients have had 3D reconstructions of their DICOM data sets, including preoperative interrogation and planning of surgery.Interrogation of the 3D images live in theatre and comparison with the surgeons' operative findings (including intraoperative ultrasound) led to the operation being abandoned in 25% of cases, adoption of an alternative surgical approach in 25% of cases, and helpful image guidance for successful resection in 50% of cases.The clinical value of the latest generation of scanners and digital imaging techniques cannot be realized unless appropriate dissemination of the images takes place. This project has succeeded in translating the image technology into a user-friendly form and delivers 3D reconstructions of patient-specific data to the "sharp end"-the surgeon undertaking the tumor resection in theatre, in a manner that allows interaction and interpretation. More time interrogating the 3D data sets preoperatively would help reduce the incidence of abandoned operations-this is part of the surgeons' learning curve. We have developed one of the first practical applications to benefit from remote visualization, and certainly the first medical visualization application of this kind.

Op3D is a new visualisation system for a surgeon. For the first time, patient-specific medical data sets can be rendered in three dimensions using a powerful compute server, with the reconstructions being delivered across the computer network in real time to the operating theatre. A laptop serves as the client computer and an easy-to-use interface has been developed for the surgeon to interact with and interrogate the patient data. This paper describes the Op3D system and presents the initial results from its use in hepatopancreatic surgery. Examples are given from both preoperative and intraoperative use.