A Causal Network-Based Markov Decision Process Model for Intervention Planning

Patients suffering from chronic diseases often receive interventions after they are discharged from hospitals for their health management. The intervention decisions can be made sequentially during the course of disease progression and treatment process, and timely and proper interventions can lead to better health condition and quality of life and reduce the rate of re-hospitalization. Moreover, to determine effective intervention plans, personalized care should be considered. However, in many cases, intervention plans are not tailored to patients’ health conditions or dynamically adjusted based on their status changes due to the lack of adequate analytical methods. To address this issue, we introduce a causal Bayesian network-based Markov decision process (CNMDP) model to dynamically evaluate the health status and effectiveness of interventions for each patient and update the corresponding intervention strategy. In such a model, a causal Bayesian network is integrated in order to predict the effect of interventions in each patient’s health status. The resulting risk at each decision point is considered in a Markov decision process model by examining diverse factors of a patient to make a personalized optimal decision on intervention planning to minimize the risk of health deterioration or readmission. To illustrate the applicability of the model and develop a specific formulation, an intervention decision model for chronic obstructive pulmonary disease (COPD) patients is presented using the proposed method. Such a model can be applied to other diseases as well to design personalized, prompt, and effective intervention plans. Note to Practitioners—Substantial efforts have been devoted to treatment and care management of chronic diseases. Particularly, there has been an initiative to design appropriate follow-up plans and interventions for postdischarge care to reduce hospital readmissions. However, many intervention plans are not patient specific and do not change with patient conditions. To mitigate such an limitation, based on identified risk factors and causal relationships, a causal Bayesian network model is introduced to predict readmission risk and evaluate intervention effectiveness. The results are then embedded into a Markov decision process to derive optimal intervention plan to minimize readmission risk. Such a process is repeated dynamically to evaluate health status and update intervention strategies. Using chronic obstructive pulmonary disease (COPD) as an example of chronic disease, a case study at a community hospital is presented to illustrate the applicability of the model.