Search

We compare the performance of four computerized methods in identifying chest x-ray reports that support acute bacterial pneumonia. Two of the computerized techniques are constructed from expert knowledge, and two learn rules and structure from data. The two machine learning systems perform as well as the expert constructed systems. All of the computerized techniques perform better than a baseline keyword search and a lay person, and perform as well as a physician. We conclude that machine learning can be used to identify chest x-ray reports that support pneumonia.

Computerized health information systems can contribute to the care received by patients in a number of ways. Not the least of these is through interactions with health care providers to modify diagnostic and therapeutic decisions. Since its beginning, developers have used the HELP hospital information system to explore computerized interventions into the medical decision making process. By their nature these interventions imply a computer-directed interaction with the physicians, nurses, and therapists involved in delivering care. In this paper we describe four different approaches to this intervention. These include: (1) processes that respond to the appearance of certain types of clinical data by issuing an alert informing caregivers of these data's presence and import, (2) programs that critique new orders and propose changes in those orders when appropriate, (3) programs that suggest new orders and procedures in response to patient data suggesting their need, and (4) applications that function by summarizing patient care data and that attempt to retrospectively assess the average or typical quality of medical decisions and therapeutic interventions made by health care providers. These approaches are illustrated with experience from the HELP system.

In this paper, we describe our clinical evaluation of the diagnostic accuracy of the renal mass diagnostic system (RMDS) and of seven physicians. To investigate the value of intravenous urography (IVU) and/or retrograde urography (RU) in diagnosing renal parenchymal tumors and tumors of the renal pelvis, RMDS and the seven physicians were tested with and without the information regarding IVU/RU at two different times. From this study we believe that RMDS can help residents in making more accurate presurgical renal mass diagnosis, and may eliminate the need for IVU/RU in the diagnosing process for a specific group of patients.

Few diagnostic decision-support systems are in routine clinical use, mainly because these systems typically require time-consuming manual data entry. This research investigated the feasibility of reducing manual data entry by integrating a stand-alone diagnostic expert system with an existing comprehensive hospital information system (HIS). A knowledge-based intervocabulary mapping technique was developed to map disparate vocabularies. The results of a retrospective study indicate that transferring clinical data from the HIS to the diagnostic expert system at the beginning of workup significantly reduces the manual data entry required for generating the correct diagnoses for patients.

The development and testing of computerized systems to assist in the diagnostic process is a time honored research activity in medical information science. The focus of the majority of the applications produced is on providing accurate diagnostic suggestions when appropriate clinical information is entered. We believe that diagnostic knowledge has a much wider range of uses than that of simply assigning diagnostic labels. Below we describe three applications which illustrate alternate uses for diagnostic systems. Applications that assist in data collection, assess the quality of medical reports, and extract relevant clinical data from natural language x-ray reports are discussed. We believe that more effort should be directed toward studying the use of diagnostic knowledge bases in processes that help plan diagnostic strategies, in quality assurance applications, and in processes that facilitate all aspects of medical communication.

In this paper we describe the evaluation of a probabilistic diagnostic system for patients with renal mass. Three inference models: Multi-membership Bayesian (MB), Minimal Diagnosis (MD) and Bayesian Network (BN), and 72 patients are used to illustrate three interrelated measures of system performance: accuracy, reliability and discriminating power. The inferencing strategies we tested demonstrated the kind of trade-offs in the performance measures that can be expected from imperfect systems. Ultimately, the purpose and expected use of a system should dictate the relative importance ascribed to different aspects of system performance.

Medical informatics could facilitate more effective analysis and use of clinical knowledge by means of expert systems. To be most effective, such systems should be constructed in a manner which is consistent with physicians' cognitive processes. Our past five years' work with a system called Iliad indicates that it provides effective medical training and education. The current research extends our previous work by using a wider array of training and test cases. We also evaluated whether training on specific cases could generalize to improved testing performance on related cases, which featured similar complaints and pathophysiologic mechanisms, but different final diagnoses. In their junior internal medicine clerkship, students (n = 100) completed 1300 Iliad training cases covering 48 diagnoses. The findings indicated improved problem solving on the specifically trained cases as well as the generalization cases. We discuss a possible training model for expert systems such as Iliad.

Iliad is a diagnostic expert system for internal medicine. Iliad's "best information" mode is used to determine the most cost-effective findings to pursue next at any stage of a work-up. The "best information" algorithm combines an information content calculation together with a cost factor. The calculations then provide a rank-ordering of the alternative patient findings according to cost-effectiveness. The authors evaluated five information content models under two different strategies. The first, the single-frame strategy, considers findings only within the context of each individual disease frame. The second, the across-frame strategy, considers the information that a single finding could provide across several diseases. The study found that (1) a version of Shannon's information model performed the best under both strategies---this finding confirms the result of a previous independent study, (2) the across-frame strategy was preferred over the single-frame strategy.

NEONATE is a prototype of an expert system for the Newborn Intensive Care Unit developed at the Primary Children's Medical Center in Salt Lake City, Utah. A pilot study was undertaken to see if the addition of radiological frames to the NEONATE software could aid attending Neonatologists to interpret chest X-ray films. A set of radiological frames was created from rules generated by a radiologist. The performance of these radiological frames was compared to the performance of other radiologists using the Kappa statistic to measure agreement. There is a good agreement between the computer's decisions and the radiologists' decisions. The radiological frames were also tested to see if they help physicians who are not trained in radiology. A system that compares the residents' interpretation and the computer's interpretation to a gold standard interpretation was developed. It shows that the computer helps the first and second year residents, but not the third year residents. This article suggests that NEONATE's interpretation of chest X-ray findings are close to the radiologists' interpretations. While NEONATE's radiological frames help novice physicians in reaching better chest X-ray interpretation, the current study suggests that they are not likely to help a Neonatologist.

Iliad is a computerized, expert system for internal medical diagnosis. The system is designed to teach diagnostic skills by means of simulated patient case presentations. We report the results of a controlled trial in which junior students were randomly assigned to received Iliad training on one of two different simulated case mixes. Each group was subsequently tested in both their "trained" and "untrained" case domain. The testing consisted of computerized, simulated patient cases for which no training feedback was provided. Outcome variables were designed to measure the students' performance on these test cases. The results indicate that students made fewer diagnostic errors and more conclusively confirmed their diagnostic hypotheses when they were tested in their trained domain. We conclude that expert systems such as Iliad can effectively teach diagnostic skills by supplementing trainees' actual case experience with computerized simulations.

Iliad is a diagnostic expert system for internal medicine. One important feature that Iliad offers is the ability to analyze a particular patient case and to determine the most cost-effective method for pursuing the work-up. Iliad's current "best information" algorithm has not been previously validated and compared to other potential algorithms. Therefore, this paper presents a comparison of four new algorithms to the current algorithm. The basis for this comparison was eighteen "vignette" cases derived from real patient cases from the University of Utah Medical Center. The results indicated that the current algorithm can be significantly improved. More promising algorithms are suggested for future investigation.

NEONATE is a prototype of an expert application for the HELP Hospital Information System. Its goal is to improve documentation in the Newborn Intensive Care Unit. The decision module of NEONATE is designed to produce an admission problem list. In this paper, the admission problem list that NEONATE generates was compared to the admission problem list of the current CETUS system for 30 patients. These were compared to a retrospectively constructed gold standard problem list. Of 101 problems in the gold standard list, 56 were on the current admission reports; 82 were found by NEONATE. NEONATE found 31 problems missed on the current admission reports; the current admission reports contained 5 problems missed by NEONATE. The current admission reports contained 9 false positives; whereas NEONATE's reports contained 27. Of the 27, 16 were caused by a single rule in NEONATE. We conclude that an expert system has great potential for improving the documentation of the patient problem list.

In domains where the types of data which are to be interpreted are relatively constrained (as in the case of specific laboratory test results), our modular data-driven approach can be very productive and well received by the clinical recipient of the data. The computer rarely surpasses the knowledge of an experts result from lack of communication, imperfect memory, oversight or multiple decision-makers caring for the same patient. In such cases, most of the alerts are immediately recognized as valid, so the need for elaborate explanations is not a high priority. On the other hand, a non-specialist is alerted to the need for additional investigation, tests or collaborative support, by the fact that a reminder or diagnosis that s/he had not previously considered, appears. In other words, for the expert, a data-driven system provides unceasing oversight in high-volume low-yielded situations where a small number of mistakes may uncommonly occur for reasons which are not related to the lack of knowledge of the provider. For the non-specialist the system suggests that the patient may have problems in a domain for which the physician needs additional support. In the present state of the art, we do not think that total reliance on the computer-contained knowledge is the ultimate source of this additional support; providing the awareness of the need may be the most important contribution. Once you know that you need help, it is usually obtainable. In a discussion about how computer systems have failed, Friedman and Gustafson made the following observation.(ABSTRACT TRUNCATED AT 250 WORDS)