Search

In population pharmacokinetic (PK) models, interindividual variability is explained by implementation of covariates in the model. The widely used forward stepwise selection method is sensitive to bias, which may lead to an incorrect inclusion of covariates. Alternatives, such as the full fixed effects model, reduce this bias but are dependent on the chosen implementation of each covariate. As the correct functional forms are unknown, this may still lead to an inaccurate selection of covariates. Machine learning (ML) techniques can potentially be used to learn the optimal functional forms for implementing covariates directly from data. A recent study suggested that using ML resulted in an improved selection of influential covariates. However, how do we select the appropriate functional form for including these covariates? In this work, we use SHapley Additive exPlanations (SHAP) to infer the relationship between covariates and PK parameters from ML models. As a case-study, we use data from 119 patients with hemophilia A receiving clotting factor VIII concentrate peri-operatively. We fit both a random forest and a XGBoost model to predict empirical Bayes estimated clearance and central volume from a base nonlinear mixed effects model. Next, we show that SHAP reveals covariate relationships which match previous findings. In addition, we can reveal subtle effects arising from combinations of covariates difficult to obtain using other methods of covariate analysis. We conclude that the proposed method can be used to extend ML-based covariate selection, and holds potential as a complete full model alternative to classical covariate analyses.

INTRODUCTION: In ageing societies, the number of older adults with complex chronic conditions (CCCs) is rapidly increasing. Care for older persons with CCCs is challenging, due to interactions between multiple conditions and their treatments. In home care and nursing homes, where most older persons with CCCs receive care, professionals often lack appropriate decision support suitable and sufficient to address the medical and functional complexity of persons with CCCs. This EU-funded project aims to develop decision support systems using high-quality, internationally standardised, routine care data to support better prognostication of health trajectories and treatment impact among older persons with CCCs. METHODS AND ANALYSIS: Real-world data from older persons aged ≥60 years in home care and nursing homes, based on routinely performed comprehensive geriatric assessments using interRAI systems collected in the past 20 years, will be linked with administrative repositories on mortality and care use. These include potentially up to 51 million care recipients from eight countries: Italy, the Netherlands, Finland, Belgium, Canada, USA, Hong Kong and New Zealand. Prognostic algorithms will be developed and validated to better predict various health outcomes. In addition, the modifying impact of pharmacological and non-pharmacological interventions will be examined. A variety of analytical methods will be used, including techniques from the field of artificial intelligence such as machine learning. Based on the results, decision support tools will be developed and pilot tested among health professionals working in home care and nursing homes. ETHICS AND DISSEMINATION: The study was approved by authorised medical ethical committees in each of the participating countries, and will comply with both local and EU legislation. Study findings will be shared with relevant stakeholders, including publications in peer-reviewed journals and presentations at national and international meetings. ispartof: BMJ Open vol:13 issue:6 pages:e072399- ispartof: location:England status: Published online

no abstract available

Finding well-defined clusters in data represents a fundamental challenge for many data-driven applications, and largely depends on good data representation. Drawing on literature regarding representation learning, studies suggest that one key characteristic of good latent representations is the ability to produce semantically mixed outputs when decoding linear interpolations of two latent representations. We propose the Mixing Consistent Deep Clustering (MCDC) method which encourages interpolations to appear realistic while adding the constraint that interpolations of two data points must look like one of the two inputs. By applying this training method to various clustering (non-)specific autoencoder models we found that using the proposed training method systematically changed the structure of learned representations of a model and it improved clustering performance for the tested ACAI, IDEC, and VAE models on the MNIST, SVHN, and CIFAR-10 datasets. These outcomes have practical implications for numerous real-world clustering tasks, as it shows that the proposed method can be added to existing autoencoders to further improve clustering performance.

While reinforcement learning (RL) has proven to be the approach of choice for tackling many complex problems, it remains challenging to develop and deploy RL agents in real-life scenarios successfully. This paper presents pH-RL (personalization in e-Health with RL), a general RL architecture for personalization to bring RL to health practice. pH-RL allows for various levels of personalization in health applications and allows for online and batch learning. Furthermore, we provide a general-purpose implementation framework that can be integrated with various healthcare applications. We describe a step-by-step guideline for the successful deployment of RL policies in a mobile application. We implemented our open-source RL architecture and integrated it with the MoodBuster mobile application for mental health to provide messages to increase daily adherence to the online therapeutic modules. We then performed a comprehensive study with human participants over a sustained period. Our experimental results show that the developed policies learn to select appropriate actions consistently using only a few days’ worth of data. Furthermore, we empirically demonstrate the stability of the learned policies during the study.

Time-series forecasting plays an important role in many domains. Boosted by the advances in Deep Learning algorithms, it has for instance been used to predict wind power for eolic energy production, stock market fluctuations, or motor overheating. In some of these tasks, we are interested in predicting accurately some particular moments which often are underrepresented in the dataset, resulting in a problem known as imbalanced regression. In the literature, while recognized as a challenging problem, limited attention has been devoted on how to handle the problem in a practical setting. In this paper, we put forward a general approach to analyze time-series forecasting problems focusing on those underrepresented moments to reduce imbalances. Our approach has been developed based on a case study in a large industrial company, which we use to exemplify the approach.

Reinforcement learning (RL) is a powerful framework for learning complex behaviors, but lacks adoption in many settings due to sample size requirements. We introduce a framework for increasing sample efficiency of RL algorithms. Our approach focuses on optimizing environment rewards with high-level instructions. These are modeled as a high-level controller over temporally extended actions known as options. These options can be looped, interleaved and partially ordered with a rich language for high-level instructions. Crucially, the instructions may be underspecified in the sense that following them does not guarantee high reward in the environment. We present an algorithm for control with these so-called option machines (OMs), discuss option selection for the partially ordered case and describe an algorithm for learning with OMs. We compare our approach in zero-shot, single- and multi-task settings in an environment with fully specified and underspecified instructions. We find that OMs perform significantly better than or comparable to the state-of-art in all environments and learning settings.

Additional file 1: Table S1. Parameter bounds. Table S2. Parameter categories. Table S3. Feature derivations. Table S4. Included features. Table S5. Hyper parameter optimization. Table S6. Feature Importance Metrics. Table S7. Patient characteristics. Table S8. ROC AUC score. Table S9. F1-score. Table S10. Sensitivity Analysis Minimal Prone Duration. Table S11. Sensitivity Analysis Cut-off Values. Table S12. Ranked Feature Importance. Table S13. Feature Importance Values. Figure S1. PaO2/FiO2 ratio difference in mmHg over time after starting prone position. Prone positions which continued for longer than 24 hours were filtered out.

Background The identification of risk factors for adverse outcomes and prolonged intensive care unit (ICU) stay in COVID-19 patients is essential for prognostication, determining treatment intensity, and resource allocation. Previous studies have determined risk factors on admission only, and included a limited number of predictors. Therefore, using data from the highly granular and multicenter Dutch Data Warehouse, we developed machine learning models to identify risk factors for ICU mortality, ventilator-free days and ICU-free days during the course of invasive mechanical ventilation (IMV) in COVID-19 patients. Methods The DDW is a growing electronic health record database of critically ill COVID-19 patients in the Netherlands. All adult ICU patients on IMV were eligible for inclusion. Transfers, patients admitted for less than 24 h, and patients still admitted at time of data extraction were excluded. Predictors were selected based on the literature, and included medication dosage and fluid balance. Multiple algorithms were trained and validated on up to three sets of observations per patient on day 1, 7, and 14 using fivefold nested cross-validation, keeping observations from an individual patient in the same split. Results A total of 1152 patients were included in the model. XGBoost models performed best for all outcomes and were used to calculate predictor importance. Using Shapley additive explanations (SHAP), age was the most important demographic risk factor for the outcomes upon start of IMV and throughout its course. The relative probability of death across age values is visualized in Partial Dependence Plots (PDPs), with an increase starting at 54 years. Besides age, acidaemia, low P/F-ratios and high driving pressures demonstrated a higher probability of death. The PDP for driving pressure showed a relative probability increase starting at 12 cmH2O. Conclusion Age is the most important demographic risk factor of ICU mortality, ICU-free days and ventilator-free days throughout the course of invasive mechanical ventilation in critically ill COVID-19 patients. pH, P/F ratio, and driving pressure should be monitored closely over the course of mechanical ventilation as risk factors predictive of these outcomes.

The work in this thesis is the result of answers that we aimed to find for clinical questions. We initially focused on critically-ill sepsis patients, but when faced with the COVID-19 pandemic in the intensive care unit in 2020, we broadened our scope to also include COVID-19 patients. In Part I of this thesis, we developed a scale to evaluate machine learning (ML) readiness. Although AI is poised to reshape medicine, routine clinical care in the ICU remains devoid of AI solutions. In Chapter 2 we hypothesized that machine learning models follow a trajectory towards clinical practice similar to aerospace technology. We found that 93% of all critical care AI models were in the prototyping and model development stage. Only two articles shadow tested their models, which means integrating a model into the electronic health system without presenting the results to clinicians, and only two publications evaluated models against clinically relevant outcomes. Secondly, we evaluated the diagnostic test accuracy of all identifiable machine learning models to predict the onset of sepsis. We found 28 papers that contained 130 models eligible for inclusion. Overall, machine learning models were able to accurately predict sepsis onset on retrospective data. However, only 6.2% of these models were implemented prospectively and only one study was implemented in clinical practice. Part II of this thesis describes the unification of clinical data from ICU patients with COVID-19. Both Chapter 4 and 5 describe the legal and technical details of the Dutch Data Warehouse (DDW), which contains 3464 patients from 25 hospitals, both from wave 1 (before the summer of 2020) and wave 2 in the Netherlands. In parallel with data collection for the DDW, we addressed several questions that came up from clinical care for COVID-19 patients. In Part III, Chapter 6, we identified the risk factors for extubation failure in COVID-19 patients. The most important risk factors for extubation failure were ventilatory characteristics, indicators of inflammation, the Glasgow Coma Scale, and body mass index. They are recognized clinically for their importance in predicting sepsis onset. The second application in Chapter 7 describes the risk factors for mortality, ICU-free days and ventilator free days in COVID-19 patients. We found that the strongest predictor of mortality in critically-ill COVID-19 patients was age. Similar to Chapter 6, we identified clinical factors that require monitoring and capturing in the EHR as they were strongly associated to patient outcomes, relative to the other parameters in the model. These factors include clinical parameters not typically present in registry datasets, such as pH, P/F ratio and driving pressure. Overall, the clinical parameters most associated with adverse outcomes may easily be recorded in EHRs and are familiar to ICU clinicians. In Part IV, we outline our implementation of clinical decision support software in clinical practice. Clinical implementation is one of the central hurdles in critical care machine learning. Chapter 8 describes the use case for clinical decision support to guide antibiotic dosing in critically-ill patients. We performed a cross-sectional survey study to assess pharmacokinetic knowledge among intensive care clinicians. Pass rates of the test were below 50% for intensivists, intensive care fellows, residents and ICU nurses. These results demonstrate the need for aiding clinicians in pharmacokinetic guided antibiotic dosing. In Chapter 9 and 10 we describe the clinical implementation of AutoKinetics, a computerized dosing advice software for four antibiotics. The software provides continuous and real-time dosing advice to clinicians at the intensive care bedside. We show that with AutoKinetics, real-time, continuous clinical decision support is feasible at the ICU bedside and can be integrated into the clinical workflow.

The enormous amounts of data make it progressively difficult for a human to assign meaning to individual data points. For example, if several users are active on different devices in a computer network, all kinds of connections are registered: browsing websites, scrolling through social media, video calling with friends, and so on. However, these `human' labels are usually not stored by the computer. And, simply due to time restrictions, it is not possible for a cyber expert to give an interpretation to all the traffic. Fortunately, over the past decades, the field of data science has grown at lightning speed. Often Machine Learning (ML) is used to extract information from all kinds of data. ML includes mathematical algorithms that learn patterns in data and that can make predictions about it. Since a computer can perform calculations extremely much faster, huge datasets can be interpreted in considerably less time. Typically ML has difficulty recognising patterns of data that is has not been able to learn from before, though. People, however, are good at assigning meaning to deviating data points by using domain knowledge. Therefore, we investigate how we can combine human knowledge and computer power to arrive at better and understandable predictions. This thesis focuses on molding the symbiosis between human and machine. We consider the following three themes: Anomaly Detection, Model Evaluation and Active Learning. Anomaly detection algorithms are widely used within network intrusion detection, the field that deals with finding cyber attacks in network data. In Chapters 2 and 3, we apply algorithms to detect malicious points in, respectively, real cyber data and data with online aeroplane bookings that contain fraudulent reservations. First, we remove redundant or uninformative data features and create new variables by using domain knowledge and by combining or standardising other features. In addition, we demonstrate how we deal with unlabelled and partially labelled data. Finally, we involve human experts by having them evaluate the results. By having the experts assess data points in a targeted manner, we efficiently obtain an indication of the predictive power of ML algorithms. Moreover, malicious data can be found faster. It is difficult to determine the predictive power of an ML model when fully labelled evaluation data is insufficiently available. Therefore, in Chapter 4 we develop a new evaluation metric that provides a good and robust estimate of the predictive power when no negative labels are available. In Chapter 5, we consider Model Evaluation more fundamentally using a common problem that arises when evaluating an algorithm. Often an evaluation metric is used to determine predictive power. For example, a metric score of 80% sounds good, but on what is this feeling based? Therefore, we introduce the Dutch Draw as a method to generate general, simple, but informative baselines. In the Active Learning (AL) paradigm, the human expert becomes an integral part of the methodology as a labelling source. AL is important in fields where labelled data is relatively scarce. In Chapters 6 and 7, we develop new AL methods for network intrusion detection. The first method, Jasmine, has a dynamic selection procedure built in. This means that unlabelled data is given meaning in a more effective manner, allowing malicious network traffic to be discovered at an earlier stage. In Chapter 7, we improve the Jasmine methodology and extend it to Plusmine. We refine the dynamic selection procedure and we include Automatic Classification (AC) that increases the speed of labelling. An important advantage of our method is that it works relatively simply and therefore requires little extra computational resources.

Reinforcement learning has shown great potential in many applications in recent years. However, many of these applications rely on simulators such as games to obtain a significant amount of interactions at a relatively low cost. Especially in the health domain data suitable for sequential decision-making is hard to obtain, while simulators are complicated to build. In this thesis, we investigate how we can use reinforcement learning for personalization in e-Health. We explored the state-of-the-art applications of reinforcement learning for personalization through a systematic literature review. Looking at the related work in the health domain, we found that reinforcement learning is an appropriate learning paradigm for personalization in general. Furthermore, we observed a lack of simulator environments and standardized datasets for the health domain. To mitigate this limitation, we developed an open-source simulation environment for e-Health problems. Using this simulator as a testbed, we proposed a cluster-based reinforcement learning approach to personalize e-Health interventions. Our approach finds similarities in behaviors and learns personalized policies optimized for long-term health behaviors. To bridge the gap towards real-life applications, we employed generative models, such as generative adversarial models, to synthesize low-level sensor data we can obtain from mobile devices in real-life. To evaluate the generated data's representativity for sequential decision-making problems, we proposed a framework consisting of two properties, structural and functional representativity. Using our realistic simulator, we used deep reinforcement learning to demonstrate that end-to-end learning on raw sensor data can improve performance. Finally, we utilized reinforcement learning for personalization for several real-world problems. We applied deep reinforcement learning for the hemodynamic optimization of patients with sepsis at the Intensive Care Unit. Finally, we applied our proposed cluster-based reinforcement learning approach for personalization to improve adherence to internet-based interventions tested in a real-life experiment using a mobile application.

With the current digitisation of our world, we have witnessed a surge in the presence and use of mobile devices. Consequently, there has been a natural increase in the use of mobile applications (apps). A category of app that has been growing in popularity is e-Health apps. However, even though popular, e-Health apps have many shortcomings that need to be addressed. Most notably, the rules and mechanisms employed by current day e-Health apps do not use the full potential of context and features they have access to. Leading to apps that are too rigid and not well tailored to the users' needs and goals. In this thesis we look at overcoming this rigidity and sub-optimal tailoring of e-Health apps. To reach this goal, we propose combining AI-based personalisation and software self-adaptation. For personalisation, we choose to use reinforcement learning (RL) as it is a good fit in providing personalisation in the e-Health domain. We explore the current state of the art by conducting a systematic literature review. With this review we identify two main weaknesses of RL: it requires a lot of data to reach an optimal policy and exploration can lead to user disengagement. To tackle the former, we propose cluster-based RL. We then further improve our proposed solution by developing an online clustering algorithm designed for e-Health. For the latter, we explore how machine learning can be used to predict user engagement. To better understand software self-adaptation in the domain of apps, we conduct a systematic literature review. In the review, we classify the current approaches and identify several shortcomings relevant to e-Health apps. Lastly, to tackle the identified shortcomings and combine personalisation and self-adaptation, we introduce a reference architecture for personalised and self-adaptive e-Health apps. We explore the benefits that said architecture can have on social sustainability and empirically evaluate an app implemented following this architecture. For the empirical evaluation two experiments were performed: a user study and a measurement-based experiment. With the user study, we better understand the effects of the implemented app on the end users' perception and usability. With the measurement-based experiment, we investigate the effects that the app has on performance and energy consumption. Our results are promising, as the user study shows improved end users' usability and no significant drawback in end users' perception as well as no perceivable increase in energy consumption or decrease in performance.