Search

In population pharmacokinetic (PK) models, interindividual variability is explained by implementation of covariates in the model. The widely used forward stepwise selection method is sensitive to bias, which may lead to an incorrect inclusion of covariates. Alternatives, such as the full fixed effects model, reduce this bias but are dependent on the chosen implementation of each covariate. As the correct functional forms are unknown, this may still lead to an inaccurate selection of covariates. Machine learning (ML) techniques can potentially be used to learn the optimal functional forms for implementing covariates directly from data. A recent study suggested that using ML resulted in an improved selection of influential covariates. However, how do we select the appropriate functional form for including these covariates? In this work, we use SHapley Additive exPlanations (SHAP) to infer the relationship between covariates and PK parameters from ML models. As a case‐study, we use data from 119 patients with hemophilia A receiving clotting factor VIII concentrate peri‐operatively. We fit both a random forest and a XGBoost model to predict empirical Bayes estimated clearance and central volume from a base nonlinear mixed effects model. Next, we show that SHAP reveals covariate relationships which match previous findings. In addition, we can reveal subtle effects arising from combinations of covariates difficult to obtain using other methods of covariate analysis. We conclude that the proposed method can be used to extend ML‐based covariate selection, and holds potential as a complete full model alternative to classical covariate analyses. [ABSTRACT FROM AUTHOR]

Objective: Machine learning techniques can be used to extract predictive models for diseases from electronic medical records (EMRs). However, the nature of EMRs makes it difficult to apply off-the-shelf machine learning techniques while still exploiting the rich content of the EMRs. In this paper, we explore the usage of a range of natural language processing (NLP) techniques to extract valuable predictors from uncoded consultation notes and study whether they can help to improve predictive performance.Methods: We study a number of existing techniques for the extraction of predictors from the consultation notes, namely a bag of words based approach and topic modeling. In addition, we develop a dedicated technique to match the uncoded consultation notes with a medical ontology. We apply these techniques as an extension to an existing pipeline to extract predictors from EMRs. We evaluate them in the context of predictive modeling for colorectal cancer (CRC), a disease known to be difficult to diagnose before performing an endoscopy.Results: Our results show that we are able to extract useful information from the consultation notes. The predictive performance of the ontology-based extraction method moves significantly beyond the benchmark of age and gender alone (area under the receiver operating characteristic curve (AUC) of 0.870 versus 0.831). We also observe more accurate predictive models by adding features derived from processing the consultation notes compared to solely using coded data (AUC of 0.896 versus 0.882) although the difference is not significant. The extracted features from the notes are shown be equally predictive (i.e. there is no significant difference in performance) compared to the coded data of the consultations.Conclusion: It is possible to extract useful predictors from uncoded consultation notes that improve predictive performance. Techniques linking text to concepts in medical ontologies to derive these predictors are shown to perform best for predicting CRC in our EMR dataset. [ABSTRACT FROM AUTHOR]

Abstract: In this paper a computational model is presented for how a desire triggers responses and feelings. The model shows how these feelings can be biased, for example due to addicting experiences in the past. Both the strength of a response and of the associated feeling result from a converging dynamic pattern modeled by reciprocal causal interactions between the two. The model has been used to conduct a number of simulation experiments under varying circumstances. Moreover, it has been evaluated by formal analysis of emerging patterns entailed by the model. Furthermore, it has been pointed out how the computational model can be applied within an ambient agent system supporting a human in not being tempted. In a simple example scenario it is shown such an ambient agent system is able to predict and assess a human’s desire state, and use this assessment to suggest alternatives to avoid falling for certain temptations. [Copyright &y& Elsevier]

Abstract: Trust dynamics can be modeled in relation to experiences. In this paper two models to represent human trust dynamics are introduced, namely a model on a cognitive level and a neural model. These models include a number of parameters, providing the possibility to express certain relations between trustees. The behavior of each of the models is further analyzed by means of simulation experiments and formal verification techniques. Thereafter, both models have been compared to see whether they can produce patterns that are comparable. As each of the models has its own specific set of parameters, with values that depend on the type of person modeled, such a comparison is non-trivial. To address this, a special comparison approach is introduced, based on mutual mirroring of the models in each other. More specifically, for a given parameter values set for one model, by an automated parameter estimation procedure the most optimal values for the parameter values of the other model are determined in order to show the same behavior. Roughly spoken the results are that the models can mirror each other up to an accuracy of around 90%. [Copyright &y& Elsevier]

Background: Reinforcement learning (RL) holds great promise for intensive care medicine given the abundant availability of data and frequent sequential decision-making. But despite the emergence of promising algorithms, RL driven bedside clinical decision support is still far from reality. Major challenges include trust and safety. To help address these issues, we introduce cross off-policy evaluation and policy restriction and show how detailed policy analysis may increase clinical interpretability. As an example, we apply these in the setting of RL to optimise ventilator settings in intubated covid-19 patients. Methods: With data from the Dutch ICU Data Warehouse and using an exhaustive hyperparameter grid search, we identified an optimal set of Dueling Double-Deep Q Network RL models. The state space comprised ventilator, medication, and clinical data. The action space focused on positive end-expiratory pressure (peep) and fraction of inspired oxygen (FiO2) concentration. We used gas exchange indices as interim rewards, and mortality and state duration as final rewards. We designed a novel evaluation method called cross off-policy evaluation (OPE) to assess the efficacy of models under varying weightings between the interim and terminal reward components. In addition, we implemented policy restriction to prevent potentially hazardous model actions. We introduce delta-Q to compare physician versus policy action quality and in-depth policy inspection using visualisations. Results: We created trajectories for 1118 intensive care unit (ICU) admissions and trained 69,120 models using 8 model architectures with 128 hyperparameter combinations. For each model, policy restrictions were applied. In the first evaluation step, 17,182/138,240 policies had good performance, but cross-OPE revealed suboptimal performance for 44% of those by varying the reward function used for evaluation. Clinical policy inspection facilitated assessment of action decisions for individual patients, including identification of action space regions that may benefit most from optimisation. Conclusion: Cross-OPE can serve as a robust evaluation framework for safe RL model implementation by identifying policies with good generalisability. Policy restriction helps prevent potentially unsafe model recommendations. Finally, the novel delta-Q metric can be used to operationalise RL models in clinical practice. Our findings offer a promising pathway towards application of RL in intensive care medicine and beyond. [ABSTRACT FROM AUTHOR]

Reinforcement Learning (RL) has recently found many applications in the healthcare domain thanks to its natural fit to clinical decision-making and ability to learn optimal decisions from observational data. A key challenge in adopting RL-based solution in clinical practice, however, is the inclusion of existing knowledge in learning a suitable solution. Existing knowledge from e.g. medical guidelines may improve the safety of solutions, produce a better balance between short- and long-term outcomes for patients and increase trust and adoption by clinicians. We present a framework for including knowledge available from medical guidelines in RL. The framework includes components for enforcing safety constraints and an approach that alters the learning signal to better balance short- and long-term outcomes based on these guidelines. We evaluate the framework by extending an existing RL-based mechanical ventilation (MV) approach with clinically established ventilation guidelines. Results from off-policy policy evaluation indicate that our approach has the potential to decrease 90-day mortality while ensuring lung protective ventilation. This framework provides an important stepping stone towards implementations of RL in clinical practice and opens up several avenues for further research. • Reinforcement learning based on observational data and existing knowledge. • Treatment advice and physiological knowledge in guidelines are injected into learner. • Ventilation case study indicates adherence to guidelines, clinician outperformance. [ABSTRACT FROM AUTHOR]

Introduction: In recent years, reinforcement learning (RL) has gained traction in the healthcare domain. In particular, RL methods have been explored for haemodynamic optimization of septic patients in the Intensive Care Unit. Most hospitals however, lack the data and expertise for model development, necessitating transfer of models developed using external datasets. This approach assumes model generalizability across different patient populations, the validity of which has not previously been tested. In addition, there is limited knowledge on safety and reliability. These challenges need to be addressed to further facilitate implementation of RL models in clinical practice.Method: We developed and validated a new reinforcement learning model for hemodynamic optimization in sepsis on the MIMIC intensive care database from the USA using a dueling double deep Q network. We then transferred this model to the European AmsterdamUMCdb intensive care database. T-Distributed Stochastic Neighbor Embedding and Sequential Organ Failure Assessment scores were used to explore the differences between the patient populations. We apply off-policy policy evaluation methods to quantify model performance. In addition, we introduce and apply a novel deep policy inspection to analyse how the optimal policy relates to the different phases of sepsis and sepsis treatment to provide interpretable insight in order to assess model safety and reliability.Results: The off-policy evaluation revealed that the optimal policy outperformed the physician policy on both datasets despite marked differences between the two patient populations and physician's policies. Our novel deep policy inspection method showed insightful results and unveiled that the model could initiate therapy adequately and adjust therapy intensity to illness severity and disease progression which indicated safe and reliable model behaviour. Compared to current physician behavior, the developed policy prefers a more liberal use of vasopressors with a more restrained use of fluid therapy in line with previous work.Conclusion: We created a reinforcement learning model for optimal bedside hemodynamic management and demonstrated model transferability between populations from the USA and Europe for the first time. We proposed new methods for deep policy inspection integrating expert domain knowledge. This is expected to facilitate progression to bedside clinical decision support for the treatment of critically ill patients. [ABSTRACT FROM AUTHOR]

The goals of this study were to examine whether machine-learning algorithms outperform multivariable logistic regression in the prediction of insufficient response to methotrexate (MTX); secondly, to examine which features are essential for correct prediction; and finally, to investigate whether the best performing model specifically identifies insufficient responders to MTX (combination) therapy. The prediction of insufficient response (3-month Disease Activity Score 28-Erythrocyte-sedimentation rate (DAS28-ESR) > 3.2) was assessed using logistic regression, least absolute shrinkage and selection operator (LASSO), random forest, and extreme gradient boosting (XGBoost). The baseline features of 355 rheumatoid arthritis (RA) patients from the "treatment in the Rotterdam Early Arthritis CoHort" (tREACH) and the U-Act-Early trial were combined for analyses. The model performances were compared using area under the curve (AUC) of receiver operating characteristic (ROC) curves, 95% confidence intervals (95% CI), and sensitivity and specificity. Finally, the best performing model following feature selection was tested on 101 RA patients starting tocilizumab (TCZ)-monotherapy. Logistic regression (AUC = 0.77 95% CI: 0.68–0.86) performed as well as LASSO (AUC = 0.76, 95% CI: 0.67–0.85), random forest (AUC = 0.71, 95% CI: 0.61 = 0.81), and XGBoost (AUC = 0.70, 95% CI: 0.61–0.81), yet logistic regression reached the highest sensitivity (81%). The most important features were baseline DAS28 (components). For all algorithms, models with six features performed similarly to those with 16. When applied to the TCZ-monotherapy group, logistic regression's sensitivity significantly dropped from 83% to 69% (p = 0.03). In the current dataset, logistic regression performed equally well compared to machine-learning algorithms in the prediction of insufficient response to MTX. Models could be reduced to six features, which are more conducive for clinical implementation. Interestingly, the prediction model was specific to MTX (combination) therapy response. [ABSTRACT FROM AUTHOR]

A variety of effective psychotherapies for depression are available, but patients who suffer from depression vary in their treatment response. Combining face-to-face therapies with internet-based elements in the sense of blended treatment is a new approach to treatment for depression. The goal of this study was to answer the following research questions: (1) What are the most important predictors determining optimal treatment allocation to treatment as usual or blended treatment? and (2) Would model-determined treatment allocation using this predictive information and the personalized advantage index (PAI)-approach result in better treatment outcomes? Bayesian model averaging (BMA) was applied to the data of a randomized controlled trial (RCT) comparing the efficacy of treatment as usual and blended treatment in depressive outpatients. Pre-treatment symptomatology and treatment expectancy predicted outcomes irrespective of treatment condition, whereas different prescriptive predictors were found. A PAI of 2.33 PHQ-9 points was found, meaning that patients who would have received the treatment that is optimal for them would have had a post-treatment PHQ-9 score that is two points lower than if they had received the treatment that is suboptimal for them. For 29% of the sample, the PAI was five or greater, which means that a substantial difference between the two treatments was predicted. The use of the PAI approach for clinical practice must be further confirmed in prospective research; the current study supports the identification of specific interventions favorable for specific patients. [ABSTRACT FROM AUTHOR]