Search

This paper describes a prototype for research to evaluate the impact of diagnostic decision support systems on the behavior of physicians. Several indices that can be used to quantify the magnitude of impact are proposed. A large medical diagnostic knowledge base in internal medicine (the Iliad knowledge base) was used in this evaluation. The impact on behavior when different inference models are run against this knowledge base is evaluated for two different case domains and physician's specialties.

The value of the computerized medical record is derived in part from the availability of medical information in a coded form accessible to manipulation by processes designed for automated decision support, medical research, and computer assistance in the management of health care delivery. To meet these needs medical reports captured and stored as natural language documents must be encoded. Below we discuss an ongoing formative process aimed at developing a natural language understanding system for chest x-ray reports. Comparative data showing the progress of this process is presented.

The Computerized Severity Index (CSI) is a commercially available scoring system for hospital inpatients. Trained abstractors review the patient's paper medical record and enter the diagnoses and relevant physiological attributes. The HELP (Health Evaluation through Logical Processing) System at LDS Hospital stores patient data in discrete codes. This paper describes the development of an automatic interface between the standalone, personal-computer-based severity system and the mainframe-based hospital information system. The interface scores patient severity without the need for manual chart review. Severity scores from the automated and manual methods were identical for 70% of 222 general medical patients scored retrospectively. An evaluation of the causes for differing scores between the two methods is presented.

A new client-server based system which is centered around a lifetime data repository (LDR) is under construction. The goal of the new system is to maintain the patient centered decision support aspects of the existing HELP* system while providing an open architecture that supports faster application development and allows execution of applications to be distributed across many computers. These goals are achieved by implementing the system with software components that are commercially available or by adhering to national and international standards for software integration. Keys to successful integration include the use of MS-DOS @, OS/2#, and UNIX Section as operating systems, Microsoft OLE 2.0 as a standard interface to the clinical database, the use of TUXEDO as a transaction/communication manager, and the use of ORACLE [symbol: see text] RDBMS as the underlying database management system.

Iliad is a large medical diagnostic system that covers more than 2000 diagnoses and 9000 findings. Due to the size and the complexity of this system, a robust knowledge representation is essential to consistently and efficiently model the medical knowledge involved. In this paper, we describe the knowledge representation currently used in Iliad and a probabilistic representation based on the Bayesian network formalism which can be derived using the information that the Iliad knowledge base contains.

The development and testing of computerized systems to assist in the diagnostic process is a time honored research activity in medical information science. The focus of the majority of the applications produced is on providing accurate diagnostic suggestions when appropriate clinical information is entered. We believe that diagnostic knowledge has a much wider range of uses than that of simply assigning diagnostic labels. Below we describe three applications which illustrate alternate uses for diagnostic systems. Applications that assist in data collection, assess the quality of medical reports, and extract relevant clinical data from natural language x-ray reports are discussed. We believe that more effort should be directed toward studying the use of diagnostic knowledge bases in processes that help plan diagnostic strategies, in quality assurance applications, and in processes that facilitate all aspects of medical communication.

In this paper we describe the evaluation of a probabilistic diagnostic system for patients with renal mass. Three inference models: Multi-membership Bayesian (MB), Minimal Diagnosis (MD) and Bayesian Network (BN), and 72 patients are used to illustrate three interrelated measures of system performance: accuracy, reliability and discriminating power. The inferencing strategies we tested demonstrated the kind of trade-offs in the performance measures that can be expected from imperfect systems. Ultimately, the purpose and expected use of a system should dictate the relative importance ascribed to different aspects of system performance.

Medical informatics could facilitate more effective analysis and use of clinical knowledge by means of expert systems. To be most effective, such systems should be constructed in a manner which is consistent with physicians' cognitive processes. Our past five years' work with a system called Iliad indicates that it provides effective medical training and education. The current research extends our previous work by using a wider array of training and test cases. We also evaluated whether training on specific cases could generalize to improved testing performance on related cases, which featured similar complaints and pathophysiologic mechanisms, but different final diagnoses. In their junior internal medicine clerkship, students (n = 100) completed 1300 Iliad training cases covering 48 diagnoses. The findings indicated improved problem solving on the specifically trained cases as well as the generalization cases. We discuss a possible training model for expert systems such as Iliad.

Iliad is a diagnostic expert system for internal medicine. Iliad's "best information" mode is used to determine the most cost-effective findings to pursue next at any stage of a work-up. The "best information" algorithm combines an information content calculation together with a cost factor. The calculations then provide a rank-ordering of the alternative patient findings according to cost-effectiveness. The authors evaluated five information content models under two different strategies. The first, the single-frame strategy, considers findings only within the context of each individual disease frame. The second, the across-frame strategy, considers the information that a single finding could provide across several diseases. The study found that (1) a version of Shannon's information model performed the best under both strategies---this finding confirms the result of a previous independent study, (2) the across-frame strategy was preferred over the single-frame strategy.

The archival database of the HELP Hospital Information System at LDS Hospital is too large to be stored on line. The archival data are important for clinical and research applications. Demountable disk packs have been used to store the archival database. This method of storage has four significant disadvantages. A virtual database was developed to overcome the limitations of this data-management scheme. This virtual database enables the transparent use of appropriate low-cost network-based storage technology to provide on-line availability of the entire archive of clinical data. The virtual database successfully resolves the problems associated with disk packs, and opens the door to enhanced use of the data for clinical and research applications.

Information management is central to modern patient care. Computerization of information management has resulted in both departmental systems which serve information needs in locations such as the Radiology Department and in hospital-wide information systems which seek to integrate management of clinical data from many departments. For each of these systems to achieve the goal of maximizing both the effectiveness of health care workers and the quality of patient care, they need to share the data that they capture. Below we discuss a variety of applications, both currently available and in the realm of research protocols, that depend on a high level of communication between Radiology Information Systems and Hospital Information Systems. These examples suggest the benefits of integrating the medically relevant data collected by all of the computer-based information systems in the hospital setting.

Iliad is a diagnostic expert system for internal medicine. One important feature that Iliad offers is the ability to analyze a particular patient case and to determine the most cost-effective method for pursuing the work-up. Iliad's current "best information" algorithm has not been previously validated and compared to other potential algorithms. Therefore, this paper presents a comparison of four new algorithms to the current algorithm. The basis for this comparison was eighteen "vignette" cases derived from real patient cases from the University of Utah Medical Center. The results indicated that the current algorithm can be significantly improved. More promising algorithms are suggested for future investigation.