Search

Deep learning (DL) models for medical image classification frequently struggle to generalize to data from outside institutions. Additional clinical data are also rarely collected to comprehensively assess and understand model performance amongst subgroups. Following the development of a single-center model to identify the lung sliding artifact on lung ultrasound (LUS), we pursued a validation strategy using external LUS data. As annotated LUS data are relatively scarce—compared to other medical imaging data—we adopted a novel technique to optimize the use of limited external data to improve model generalizability. Externally acquired LUS data from three tertiary care centers, totaling 641 clips from 238 patients, were used to assess the baseline generalizability of our lung sliding model. We then employed our novel Threshold-Aware Accumulative Fine-Tuning (TAAFT) method to fine-tune the baseline model and determine the minimum amount of data required to achieve predefined performance goals. A subgroup analysis was also performed and Grad-CAM++ explanations were examined. The final model was fine-tuned on one-third of the external dataset to achieve 0.917 sensitivity, 0.817 specificity, and 0.920 area under the receiver operator characteristic curve (AUC) on the external validation dataset, exceeding our predefined performance goals. Subgroup analyses identified LUS characteristics that most greatly challenged the model’s performance. Grad-CAM++ saliency maps highlighted clinically relevant regions on M-mode images. We report a multicenter study that exploits limited available external data to improve the generalizability and performance of our lung sliding model while identifying poorly performing subgroups to inform future iterative improvements. This approach may contribute to efficiencies for DL researchers working with smaller quantities of external validation data.

Clinical workplace-based learning is essential for undergraduate health professions, requiring adequate training and timely feedback. While the Mini-CEX is a well-known tool for workplace-based learning, its written paper assessment can be cumbersome in a clinical setting. We conducted a utility analysis to assess the effectiveness of an adapted Mini-CEX implemented as a mobile device WebApp for clinical practice assessment. We included 24 clinical teachers from 11 different clinical placements and 95 undergraduate physical therapy students. The adapted Mini-CEX was tailored to align with the learning outcomes of clinical practice requirements and made accessible through a WebApp for mobile devices. To ensure the validity of the content, we conducted a Delphi panel. Throughout the semester, the students were assessed four times while interacting with patients. We evaluated the utility of the adapted Mini-CEX based on validity, reliability, acceptability, cost, and educational impact. We performed factor analysis and assessed the psychometric properties of the adapted tool. Additionally, we conducted two focus groups and analyzed the themes from the discussions to explore acceptability and educational impact. The adapted Mini-CEX consisted of eight validated items. Our analysis revealed that the tool was unidimensional and exhibited acceptable reliability (0.78). The focus groups highlighted two main themes: improving learning assessment and the perceived impact on learning. Overall, the eight-item Mini-CEX WebApp proved to be a valid, acceptable, and reliable instrument for clinical practice assessment in workplace-based learning settings for undergraduate physiotherapy students. We anticipate that our adapted Mini-CEX WebApp can be easily implemented across various clinical courses and disciplines.

Introduction: The clinical presentation of coronavirus disease 2019 (COVID-19) overlaps with many other common cold and influenza viruses. Identifying patients with a higher probability of infection becomes crucial in settings with limited access to testing. We developed a prediction instrument to assess the likelihood of a positive polymerase chain reaction (PCR) test, based solely on clinical variables that can be determined within the time frame of an emergency department (ED) patient encounter. Methods: We derived and prospectively validated a model to predict SARS-CoV-2 PCR positivity in patients visiting the ED with symptoms consistent with the disease. Results: Our model was based on 617 ED visits. In the derivation cohort, the median age was 36 years, 43% were men, and 9% had a positive result. The median time to testing from the onset of initial symptoms was four days (interquartile range [IQR]: 2–5 days, range 0–23 days), and 91% of all patients were discharged home. The final model based on a multivariable logistic regression included a history of close contact (adjusted odds ratio [AOR] 2.47, 95% confidence interval [CI], 1.29–4.7); fever (AOR 3.63, 95% CI, 1.931–6.85); anosmia or dysgeusia (AOR 9.7, 95% CI, 2.72–34.5); headache (AOR 1.95, 95% CI, 1.06–3.58), myalgia (AOR 2.6, 95% CI, 1.39–4.89); and dry cough (AOR 1.93, 95% CI, 1.02–3.64). The area under the curve (AUC) from the derivation cohort was 0.79 (95% CI, 0.73–0.85) and AUC 0.7 (95% CI, 0.61–0.75) in the validation cohort (N = 379). Conclusion: We developed and validated a clinical tool to predict SARS-CoV-2 PCR positivity in patients presenting to the ED to assist with patient disposition in environments where COVID-19 tests or timely results are not readily available.