Search

Markov decision processes (MDPs) have found success in many application areas that involve sequential decision making under uncertainty, including the evaluation and design of treatment and screening protocols for medical decision making. However, the data used to parameterize the model can influence what policies are recommended, and multiple competing data sources are common in many application areas, including medicine. In this article, we introduce the Multi-model Markov decision process (MMDP) which generalizes a standard MDP by allowing for multiple models of the rewards and transition probabilities. Solution of the MMDP generates a single policy that maximizes the weighted performance over all models. This approach allows the decision maker to explicitly trade-off conflicting sources of data while generating a policy of the same level of complexity for models that only consider a single source of data. We study the structural properties of this problem and show that it is at least NP-hard. We develop exact methods and fast approximation methods supported by error bounds. Finally, we illustrate the effectiveness and the scalability of our approach using a case study in preventative blood pressure and cholesterol management that accounts for conflicting published cardiovascular risk models.

This article addresses an appointment scheduling problem in which the server responds to congestion of the service system. Using waiting time as a proxy for how far behind schedule the server is running, we characterize the congestion-induced behavior of the server as a function of a customer’s waiting time. Decision variables are the scheduled arrival times for a specific sequence of customers. The objective of our model is to minimize a weighted cost incurred for a customer’s waiting time, server overtime and server speedup in response to congestion. We provide alternative formulations of this problem as a Simulation Optimization (SO) model and a Stochastic Integer Programming (SIP) model, respectively. We show that the SIP model can solve moderate-sized instances exactly under certain assumptions about a server′s response to congestion. We further show that the SO model achieves near-optimal solutions for moderate-sized problems while also being able to scale up to much larger problem instances. We present theoretical results for both models and we carry out a series of experiments to illustrate the characteristics of the optimal schedules and to measure the importance of accounting for a server′s response to congestion when scheduling appointments using a case study for an outpatient clinic at a large medical center. Finally, we summarize the most important managerial insights obtained from this study.

Surgical services are large revenue sources that account for a large portion of hospital expenses. Thus, efficient resource allocation is crucial in this system; however, this is a challenging problem, in part due to the interaction of the different stages of the surgery delivery system and the uncertainty of surgery and recovery durations. This article focuses on single-day in-patient elective surgery scheduling considering surgeons, operating rooms (ORs), and the post-anesthesia care unit (recovery). We propose a mixed-integer programming formulation of this problem and then present a fast two-phase heuristic: phase 1 is used for determining the number of ORs to open for the day and surgeon-to-OR assignments, and phase 2 is used for surgical case sequencing. Both phases have provable worst-case performance guarantees and excellent average case performance. We evaluate schedules under uncertainty using a discrete-event simulation model based on data provided by a mid-sized hospital. We show that the fast and easy-to-implement two-phase heuristic performs extremely well, in both deterministic and stochastic settings. The new methods developed reduce the computational barriers to implementation and demonstrate that hospitals can realize substantial benefits without resorting to sophisticated optimization software implementations.