Search

Abstract Background Traditional clustering techniques are typically restricted to either continuous or categorical variables. However, most real-world clinical data are mixed type. This study aims to introduce a clustering technique specifically designed for datasets containing both continuous and categorical variables to offer better clustering compatibility, adaptability, and interpretability than other mixed type techniques. Methods This paper proposed a modified Gower distance incorporating feature importance as weights to maintain equal contributions between continuous and categorical features. The algorithm (DAFI) was evaluated using five simulated datasets with varying proportions of important features and real-world datasets from the 2011–2014 National Health and Nutrition Examination Survey (NHANES). Effectiveness was demonstrated through comparisons with 13 clustering techniques. Clustering performance was assessed using the adjusted Rand index (ARI) for accuracy in simulation studies and the silhouette score for cohesion and separation in NHANES. Additionally, multivariable logistic regression estimated the association between periodontitis (PD) and cardiovascular diseases (CVDs), adjusting for clusters in NHANES. Results In simulation studies, the DAFI-Gower algorithm consistently performs better than baseline methods according to the adjusted Rand index in settings investigated, especially on datasets with more redundant features. In NHANES, 3,760 people were analyzed. DAFI-Gower achieves the highest silhouette score (0.79). Four distinct clusters with diverse health profiles were identified. By incorporating feature importance, we found that cluster formations were more strongly influenced by CVD-related factors. The association between periodontitis and cardiovascular diseases, after adjusting for clusters, reveals significant insights (adjusted OR 1.95, 95% CI 1.50 to 2.55, p = 0.012), highlighting severe periodontitis as a potential risk factor for cardiovascular diseases. Conclusions DAFI performed better than classic clustering baselines on both simulated and real-world datasets. It effectively captures cluster characteristics by considering feature importance, which is crucial in clinical settings where many variables may be similar or irrelevant. We envisage that DAFI offers an effective solution for mixed type clustering.

BackgroundCognitive biases in clinical decision-making significantly contribute to errors in diagnosis and suboptimal patient outcomes. Addressing these biases presents a formidable challenge in the medical field. ObjectiveThis study aimed to explore the role of large language models (LLMs) in mitigating these biases through the use of the multi-agent framework. We simulate the clinical decision-making processes through multi-agent conversation and evaluate its efficacy in improving diagnostic accuracy compared with humans. MethodsA total of 16 published and unpublished case reports where cognitive biases have resulted in misdiagnoses were identified from the literature. In the multi-agent framework, we leveraged GPT-4 (OpenAI) to facilitate interactions among different simulated agents to replicate clinical team dynamics. Each agent was assigned a distinct role: (1) making the final diagnosis after considering the discussions, (2) acting as a devil’s advocate to correct confirmation and anchoring biases, (3) serving as a field expert in the required medical subspecialty, (4) facilitating discussions to mitigate premature closure bias, and (5) recording and summarizing findings. We tested varying combinations of these agents within the framework to determine which configuration yielded the highest rate of correct final diagnoses. Each scenario was repeated 5 times for consistency. The accuracy of the initial diagnoses and the final differential diagnoses were evaluated, and comparisons with human-generated answers were made using the Fisher exact test. ResultsA total of 240 responses were evaluated (3 different multi-agent frameworks). The initial diagnosis had an accuracy of 0% (0/80). However, following multi-agent discussions, the accuracy for the top 2 differential diagnoses increased to 76% (61/80) for the best-performing multi-agent framework (Framework 4-C). This was significantly higher compared with the accuracy achieved by human evaluators (odds ratio 3.49; P=.002). ConclusionsThe multi-agent framework demonstrated an ability to re-evaluate and correct misconceptions, even in scenarios with misleading initial investigations. In addition, the LLM-driven, multi-agent conversation framework shows promise in enhancing diagnostic accuracy in diagnostically challenging medical scenarios.

Abstract Artificial intelligence (AI) has been extensively researched in medicine, but its practical application remains limited. Meanwhile, there are various disparities in existing AI-enabled clinical studies, which pose a challenge to global health equity. In this study, we conducted an in-depth analysis of the geo-economic distribution of 159 AI-enabled clinical studies, as well as the gender disparities among these studies. We aim to reveal these disparities from a global literature perspective, thus highlighting the need for equitable access to medical AI technologies.

Artificial intelligence (AI) and machine learning (ML) have potential to revolutionize emergency medical care by enhancing triage systems, improving diagnostic accuracy, refining prognostication, and optimizing various aspects of clinical care. However, as clinicians often lack AI expertise, they might perceive AI as a “black box,” leading to trust issues. To address this, “explainable AI,” which teaches AI functionalities to end-users, is important. This review presents the definitions, importance, and role of explainable AI, as well as potential challenges in emergency medicine. First, we introduce the terms explainability, interpretability, and transparency of AI models. These terms sound similar but have different roles in discussion of AI. Second, we indicate that explainable AI is required in clinical settings for reasons of justification, control, improvement, and discovery and provide examples. Third, we describe three major categories of explainability: pre-modeling explainability, interpretable models, and post-modeling explainability and present examples (especially for post-modeling explainability), such as visualization, simplification, text justification, and feature relevance. Last, we show the challenges of implementing AI and ML models in clinical settings and highlight the importance of collaboration between clinicians, developers, and researchers. This paper summarizes the concept of “explainable AI” for emergency medicine clinicians. This review may help clinicians understand explainable AI in emergency contexts.

Abstract Artificial intelligence (AI) has demonstrated the ability to extract insights from data, but the fairness of such data-driven insights remains a concern in high-stakes fields. Despite extensive developments, issues of AI fairness in clinical contexts have not been adequately addressed. A fair model is normally expected to perform equally across subgroups defined by sensitive variables (e.g., age, gender/sex, race/ethnicity, socio-economic status, etc.). Various fairness measurements have been developed to detect differences between subgroups as evidence of bias, and bias mitigation methods are designed to reduce the differences detected. This perspective of fairness, however, is misaligned with some key considerations in clinical contexts. The set of sensitive variables used in healthcare applications must be carefully examined for relevance and justified by clear clinical motivations. In addition, clinical AI fairness should closely investigate the ethical implications of fairness measurements (e.g., potential conflicts between group- and individual-level fairness) to select suitable and objective metrics. Generally defining AI fairness as “equality” is not necessarily reasonable in clinical settings, as differences may have clinical justifications and do not indicate biases. Instead, “equity” would be an appropriate objective of clinical AI fairness. Moreover, clinical feedback is essential to developing fair and well-performing AI models, and efforts should be made to actively involve clinicians in the process. The adaptation of AI fairness towards healthcare is not self-evident due to misalignments between technical developments and clinical considerations. Multidisciplinary collaboration between AI researchers, clinicians, and ethicists is necessary to bridge the gap and translate AI fairness into real-life benefits.

Background This study aimed to describe the clinical characteristics and epidemiology of heat-related illnesses presented at a tertiary hospital in Singapore. Method This retrospective observational study utilized electronic health records from the emergency department (ED) of Singapore General Hospital. Patients primarily diagnosed with heat-related illnesses from 2008 to 2020, were analyzed. The annual number of cases and the temporal trend of heat-related illnesses were described, along with the characteristics of patients. Results Among the 1,833,908 patients registered at the ED, 426 patients had diagnoses of heat-related illnesses. The median age (interquartile range [IQR]) was 32 (22, 43) years, with 303 (71%) being male and 201 (47%) not being Singapore citizens. The annual number of cases ranged from a minimum of 21 in 2019 to a maximum of 49 cases in 2015, except for four cases in 2020. Heat-related illnesses occurred most commonly on Sundays (233, 55%) and in December (97, 23%). A significant number of cases in December coincided with the day of an annually organized marathon (91, 21%). Approximately half of the total cases required hospitalization, with 183 (43%) admitted to the general ward and 31 (7%) admitted to the intensive care unit. The 30-day mortality was 2 (0.5%). Conclusion This descriptive study indicated the unique characteristics of heat-related illness in a tertiary care hospital in Singapore. These local contexts may be valuable to consider preventative measures for heat-related illness, in preparation for future extreme weather events related to climate change.

Background: Federated learning (FL) holds promise for safeguarding data privacy in healthcare collaborations. While the term “FL” was originally coined by the engineering community, the statistical field has also developed privacy-preserving algorithms, though these are less recognized. Our goal was to bridge this gap with the first comprehensive comparison of FL frameworks from both domains. Methods: We assessed 7 FL frameworks, encompassing both engineering-based and statistical FL algorithms, and compared them against local and centralized modeling of logistic regression and least absolute shrinkage and selection operator (Lasso). Our evaluation utilized both simulated data and real-world emergency department data, focusing on comparing both estimated model coefficients and the performance of model predictions. Results: The findings reveal that statistical FL algorithms produce much less biased estimates of model coefficients. Conversely, engineering-based methods can yield models with slightly better prediction performance, occasionally outperforming both centralized and statistical FL models. Conclusion: This study underscores the relative strengths and weaknesses of both types of methods, providing recommendations for their selection based on distinct study characteristics. Furthermore, we emphasize the critical need to raise awareness of and integrate these methods into future applications of FL within the healthcare domain.

Abstract Background Risk prediction models are useful tools in clinical decision-making which help with risk stratification and resource allocations and may lead to a better health care for patients. AutoScore is a machine learning–based automatic clinical score generator for binary outcomes. This study aims to expand the AutoScore framework to provide a tool for interpretable risk prediction for ordinal outcomes. Methods The AutoScore-Ordinal framework is generated using the same 6 modules of the original AutoScore algorithm including variable ranking, variable transformation, score derivation (from proportional odds models), model selection, score fine-tuning, and model evaluation. To illustrate the AutoScore-Ordinal performance, the method was conducted on electronic health records data from the emergency department at Singapore General Hospital over 2008 to 2017. The model was trained on 70% of the data, validated on 10% and tested on the remaining 20%. Results This study included 445,989 inpatient cases, where the distribution of the ordinal outcome was 80.7% alive without 30-day readmission, 12.5% alive with 30-day readmission, and 6.8% died inpatient or by day 30 post discharge. Two point-based risk prediction models were developed using two sets of 8 predictor variables identified by the flexible variable selection procedure. The two models indicated reasonably good performance measured by mean area under the receiver operating characteristic curve (0.758 and 0.793) and generalized c-index (0.737 and 0.760), which were comparable to alternative models. Conclusion AutoScore-Ordinal provides an automated and easy-to-use framework for development and validation of risk prediction models for ordinal outcomes, which can systematically identify potential predictors from high-dimensional data.

Summary: The AutoScore framework can automatically generate data-driven clinical scores in various clinical applications. Here, we present a protocol for developing clinical scoring systems for binary, survival, and ordinal outcomes using the open-source AutoScore package. We describe steps for package installation, detailed data processing and checking, and variable ranking. We then explain how to iterate through steps for variable selection, score generation, fine-tuning, and evaluation to generate understandable and explainable scoring systems using data-driven evidence and clinical knowledge.For complete details on the use and execution of this protocol, please refer to Xie et al. (2020),1 Xie et al. (2022)2, Saffari et al. (2022)3 and the online tutorial https://nliulab.github.io/AutoScore/. : Publisher’s note: Undertaking any experimental protocol requires adherence to local institutional guidelines for laboratory safety and ethics.

Abstract Background Despite the ease of interpretation and communication of a risk ratio (RR), and several other advantages in specific settings, the odds ratio (OR) is more commonly reported in epidemiological and clinical research. This is due to the familiarity of the logistic regression model for estimating adjusted ORs from data gathered in a cross-sectional, cohort or case-control design. The preservation of the OR (but not RR) in case-control samples has contributed to the perception that it is the only valid measure of relative risk from case-control samples. For cohort or cross-sectional data, a method known as ‘doubling-the-cases’ provides valid estimates of RR and an expression for a robust standard error has been derived, but is not available in statistical software packages. Methods In this paper, we first describe the doubling-of-cases approach in the cohort setting and then extend its application to case-control studies by incorporating sampling weights and deriving an expression for a robust standard error. The performance of the estimator is evaluated using simulated data, and its application illustrated in a study of neonatal jaundice. We provide an R package that implements the method for any standard design. Results Our work illustrates that the doubling-of-cases approach for estimating an adjusted RR from cross-sectional or cohort data can also yield valid RR estimates from case-control data. The approach is straightforward to apply, involving simple modification of the data followed by logistic regression analysis. The method performed well for case-control data from simulated cohorts with a range of prevalence rates. In the application to neonatal jaundice, the RR estimates were similar to those from relative risk regression, whereas the OR from naive logistic regression overestimated the RR despite the low prevalence of the outcome. Conclusions By providing an R package that estimates an adjusted RR from cohort, cross-sectional or case-control studies, we have enabled the method to be easily implemented with familiar software, so that investigators are not limited to reporting an OR and can examine the RR when it is of interest.

Summary: Background: Return of spontaneous circulation (ROSC) before arrival at the emergency department is an early indicator of successful resuscitation in out-of-hospital cardiac arrest (OHCA). Several ROSC prediction scores have been developed with European cohorts, with unclear applicability in Asian settings. We aimed to develop an interpretable prehospital ROSC (P-ROSC) score for ROSC prediction based on patients with OHCA in Asia. Methods: This retrospective study examined patients who suffered from OHCA between Jan 1, 2009 and Jun 17, 2018 using data recorded in the Pan-Asian Resuscitation Outcomes Study (PAROS) registry. AutoScore, an interpretable machine learning framework, was used to develop P-ROSC. On the same cohort, the P-ROSC was compared with two clinical scores, the RACA and the UB-ROSC. The predictive power was evaluated using the area under the curve (AUC) in the receiver operating characteristic analysis. Findings: 170,678 cases were included, of which 14,104 (8.26%) attained prehospital ROSC. The P-ROSC score identified a new variable, prehospital drug administration, which was not included in the RACA score or the UB-ROSC score. Using only five variables, the P-ROSC score achieved an AUC of 0.806 (95% confidence interval [CI] 0.799–0.814), outperforming both RACA and UB-ROSC with AUCs of 0.773 (95% CI 0.765–0.782) and 0.728 (95% CI 0.718–0.738), respectively. Interpretation: The P-ROSC score is a practical and easily interpreted tool for predicting the probability of prehospital ROSC. Funding: This research received funding from SingHealth Duke-NUS ACP Programme Funding (15/FY2020/P2/06-A79).

Risk scores are widely used for clinical decision making and commonly generated from logistic regression models. Machine-learning-based methods may work well for identifying important predictors to create parsimonious scores, but such 'black box' variable selection limits interpretability, and variable importance evaluated from a single model can be biased. We propose a robust and interpretable variable selection approach using the recently developed Shapley variable importance cloud (ShapleyVIC) that accounts for variability in variable importance across models. Our approach evaluates and visualizes overall variable contributions for in-depth inference and transparent variable selection, and filters out non-significant contributors to simplify model building steps. We derive an ensemble variable ranking from variable contributions across models, which is easily integrated with an automated and modularized risk score generator, AutoScore, for convenient implementation. In a study of early death or unplanned readmission after hospital discharge, ShapleyVIC selected 6 variables from 41 candidates to create a well-performing risk score, which had similar performance to a 16-variable model from machine-learning-based ranking. Our work contributes to the recent emphasis on interpretability of prediction models for high-stakes decision making, providing a disciplined solution to detailed assessment of variable importance and transparent development of parsimonious clinical risk scores.

Abstract Background Diabetes mellitus (DM) is one of the most common chronic diseases. Individuals with DM are more likely to be hospitalised and stay longer than those without DM. Inpatient hypoglycemia and hyperglycemia, which are associated with adverse outcomes, are common, but can be prevented through hospital quality improvement programs. Methods We designed a multi-faceted intervention program with the aim of reducing inpatient hypoglycemia and hyperglycemia. This was implemented over seven phases between September 2013 to January 2016, and covered all the non-critical care wards in a tertiary hospital. The program represented a pragmatic approach that leveraged on existing resources and infrastructure within the hospital. We calculated glucometric outcomes in June to August 2016 and compared them with those in June to August 2013 to assess the overall effectiveness of the program. We used regression models with generalised estimating equations to adjust for potential confounders and account for correlations of repeated outcomes within patients and admissions. Results We observed significant reductions in patient-days affected by hypoglycemia (any glucose reading 14 mmol/L: OR = 0.84, 95% CI: 0.71 to 0.99, p = 0.041). Similar findings were observed for admission-level hypoglycemia and hyperglycemia. Further analyses suggested that these reductions started to occur four to 6 months post-implementation. Conclusions Our program was associated with sustained improvements in clinically relevant outcomes. Our described intervention could be feasibly implemented by other secondary and tertiary care hospitals by leveraging on existing infrastructure and work force.

Abstract Background The change in two measurements of a continuous outcome can be modelled directly with a linear regression model, or indirectly with a random effects model (REM) of the individual measurements. These methods are susceptible to model misspecifications, which are commonly addressed by applying monotonic transformations (e.g., Box-Cox transformation) to the outcomes. However, transforming the outcomes complicates the data analysis, especially when variable selection is involved. We propose a robust alternative through a novel application of the conditional probit (cprobit) model. Methods The cprobit model analyzes the ordered outcomes within each subject, making the estimate invariant to monotonic transformation on the outcome. By scaling the estimate from the cprobit model, we obtain the exposure effect on the change in the observed or Box-Cox transformed outcome, pending the adequacy of the normality assumption on the raw or transformed scale. Results Using simulated data, we demonstrated a similar good performance of the cprobit model and REM with and without transformation, except for some bias from both methods when the Box-Cox transformation was applied to scenarios with small sample size and strong effects. Only the cprobit model was robust to skewed subject-specific intercept terms when a Box-Cox transformation was used. Using two real datasets from the breast cancer and inpatient glycemic variability studies which utilize electronic medical records, we illustrated the application of our proposed robust approach as a seamless three-step workflow that facilitates the use of Box-Cox transformation to address non-normality with a common underlying model. Conclusions The cprobit model provides a seamless and robust inference on the change in continuous outcomes, and its three-step workflow is implemented in an R package for easy accessibility.

Summary: Background: Emergency readmission poses an additional burden on both patients and healthcare systems. Risk stratification is the first step of transitional care interventions targeted at reducing readmission. To accurately predict the short- and intermediate-term risks of readmission and provide information for further temporal risk stratification, we developed and validated an interpretable machine learning risk scoring system. Methods: In this retrospective study, all emergency admission episodes from January 1st 2009 to December 31st 2016 at a tertiary hospital in Singapore were assessed. The primary outcome was time to emergency readmission within 90 days post discharge. The Score for Emergency ReAdmission Prediction (SERAP) tool was derived via an interpretable machine learning-based system for time-to-event outcomes. SERAP is six-variable survival score, and takes the number of emergency admissions last year, age, history of malignancy, history of renal diseases, serum creatinine level, and serum albumin level during index admission into consideration. Findings: A total of 293,589 ED admission episodes were finally included in the whole cohort. Among them, 203,748 episodes were included in the training cohort, 50,937 episodes in the validation cohort, and 38,904 in the testing cohort. Readmission within 90 days was documented in 80,213 (27.3%) episodes, with a median time to emergency readmission of 22 days (Interquartile range: 8-47). For different time points, the readmission rates observed in the whole cohort were 6.7% at 7 days, 10.6% at 14 days, 13.6% at 21 days, 16.4% at 30 days, and 23.0% at 60 days. In the testing cohort, the SERAP achieved an integrated area under the curve of 0.737 (95% confidence interval: 0.730-0.743). For a specific 30-day readmission prediction, SERAP outperformed the LACE index (Length of stay, Acuity of admission, Charlson comorbidity index, and Emergency department visits in past six months) and the HOSPITAL score (Hemoglobin at discharge, discharge from an Oncology service, Sodium level at discharge, Procedure during the index admission, Index Type of admission, number of Admissions during the last 12 months, and Length of stay). Besides 30-day readmission, SERAP can predict readmission rates at any time point during the 90-day period. Interpretation: Better performance in risk prediction was achieved by the SERAP than other existing scores, and accurate information about time to emergency readmission was generated for further temporal risk stratification and clinical decision-making. In the future, external validation studies are needed to evaluate the SERAP at different settings and assess their real-world performance. Funding: This study was supported by the Singapore National Medical Research Council under the PULSES Center Grant, and Duke-NUS Medical School.

Summary: Background: Bystander cardiopulmonary resuscitation (BCPR) is a critical component of the 'chain of survival' in reducing mortality among out-of-hospital cardiac arrest (OHCA) victims. Inconsistent findings on gender disparities among adult recipients of layperson BCPR have been reported in the literature. We aimed to fill this knowledge gap by investigating the extent of gender disparities in a cross-national setting within Pan-Asian communities. Methods: We utilised data collected from the Pan-Asian Resuscitation Outcomes Study (PAROS), an international, multicentre, prospective study conducted between 2009 and 2018. We included all OHCA cases with non-traumatic arrest aetiology transported by emergency medical services and excluded study sites that did not consistently collect information about the location of cardiac arrest. Logistic regression was used to analyse the association between gender and BCPR, stratified by location. Findings: We analysed a cohort of 56,192 OHCA cases with an overall BCPR rate of 36.2% (20,329/56,192). At public locations, the BCPR rate was 31.2% (631/2022) for female and 36.4% (3235/8892) for male OHCA victims; while at home, the rate was 38.3% (6838/17,842) for females and 35.1% (9625/27,436) for males. Controlling for site differences and several factors in multivariable logistic regression, we found females less likely to receive BCPR than males in public locations (odds ratio [OR]=0.89, 95% confidence interval [CI]: 0.70–0.99), but more likely to receive BCPR at home (OR=1.16, 95% CI: 1.11–1.21). Interpretation: In Pan-Asian communities, gender differences exist in adult recipients of BCPR and differ between home and public locations. Future studies should account for additional information on bystanders and societal factors to identify targets for interventions. Funding: The study was supported by grants from the National Medical Research Council (NMRC/CSA/0049/2013) and Laerdal Foundation (20040).

Abstract Background Although criticisms regarding the dichotomisation of continuous variables are well known, applying logit model to dichotomised outcomes is the convention because the odds ratios are easily obtained and they approximate the relative risks (RRs) for rare events. Methods To avoid dichotomisation when estimating RR, the marginal standardisation method that transforms estimates from logit or probit model to RR estimate is extended to include estimates from linear model in the transformation. We conducted a simulation study to compare the statistical properties of the estimates from: (i) marginal standardisation method between models for continuous (i.e., linear model) and dichotomised outcomes (i.e., logit or probit model), and (ii) marginal standardisation method and distributional approach (i.e., marginal mean method) applied to linear model. We also compared the diagnostic test for probit, logit and linear models. For the real dataset analysis, we applied these analytical approaches to assess the management of inpatient hyperglycaemia in a pilot intervention study. Results Although the RR estimates from the marginal standardisation method were generally unbiased for all models in the simulation study, the marginal standardisation method for linear model provided estimates with higher precision and power than logit or probit model, especially when the baseline risks were at the extremes. When comparing approaches that avoid dichotomisation, RR estimates from these approaches had comparable performance. Assessing the assumption of error distribution was less powerful for logit or probit model via link test when compared with diagnostic test for linear model. After accounting for multiple thresholds representing varying levels of severity in hyperglycaemia, marginal standardisation method for linear model provided stronger evidence of reduced hyperglycaemia risk after intervention in the real dataset analysis although the RR estimates were similar across various approaches. Conclusions When compared with approaches that do not avoid dichotomisation, the RR estimated from linear model is more precise and powerful, and the diagnostic test from linear model is more powerful in detecting mis-specified error distributional assumption than the diagnostic test from logit or probit model. Our work describes and assesses the methods available to analyse data involving studies of continuous outcomes with binary representations.

Abstract Background To quantify temporal trends in age-standardized rates of disease, the convention is to fit a linear regression model to log-transformed rates because the slope term provides the estimated annual percentage change. However, such log-transformation is not always appropriate. Methods We propose an alternative method using the rank-ordered logit (ROL) model that is indifferent to log-transformation. This method quantifies the temporal trend using odds, a quantity commonly used in epidemiology, and the log-odds corresponds to the scaled slope parameter estimate from linear regression. The ROL method can be implemented by using the commands for proportional hazards regression in any standard statistical package. We apply the ROL method to estimate temporal trends in age-standardized cancer rates worldwide using the cancer incidence data from the Cancer Incidence in Five Continents plus (CI5plus) database for the period 1953 to 2007 and compare the estimates to their scaled counterparts obtained from linear regression with and without log-transformation. Results We found a strong concordance in the direction and significance of the temporal trends in cancer incidence estimated by all three approaches, and illustrated how the estimate from the ROL model provides a measure that is comparable to a scaled slope parameter estimated from linear regression. Conclusions Our method offers an alternative approach for quantifying temporal trends in incidence or mortality rates in a population that is invariant to transformation, and whose estimate of trend agrees with the scaled slope from a linear regression model.

Abstract Background Regular and timely monitoring of blood glucose (BG) levels in hospitalized patients with diabetes mellitus is crucial to optimizing inpatient glycaemic control. However, methods to quantify timeliness as a measurement of quality of care are lacking. We propose an analytical approach that utilizes BG measurements from electronic records to assess adherence to an inpatient BG monitoring protocol in hospital wards. Methods We applied our proposed analytical approach to electronic records obtained from 24 non-critical care wards in November and December 2013 from a tertiary care hospital in Singapore. We applied distributional analytics to evaluate daily adherence to BG monitoring timings. A one-sample Kolmogorov-Smirnov (1S-KS) test was performed to test daily BG timings against non-adherence represented by the uniform distribution. This test was performed among wards with high power, determined through simulation. The 1S-KS test was coupled with visualization via the cumulative distribution function (cdf) plot and a two-sample Kolmogorov-Smirnov (2S-KS) test, enabling comparison of the BG timing distributions between two consecutive days. We also applied mixture modelling to identify the key features in daily BG timings. Results We found that 11 out of the 24 wards had high power. Among these wards, 1S-KS test with cdf plots indicated adherence to BG monitoring protocols. Integrating both 1S-KS and 2S-KS information within a moving window consisting of two consecutive days did not suggest frequent potential change from or towards non-adherence to protocol. From mixture modelling among wards with high power, we consistently identified four components with high concentration of BG measurements taken before mealtimes and around bedtime. This agnostic analysis provided additional evidence that the wards were adherent to BG monitoring protocols. Conclusions We demonstrated the utility of our proposed analytical approach as a monitoring tool. It provided information to healthcare providers regarding the timeliness of daily BG measurements. From the real data application, there were empirical evidences suggesting adherence of BG timings to protocol among wards with adequate power for assessing timeliness. Our approach is extendable to other areas of healthcare where timeliness of patient care processes is important.