Using word clouds to Re-envision letters of recommendation for residency applicants

OBJECTIVES/HYPOTHESIS To develop a "word cloud"-based visual letter of recommendation (VLOR) and to evaluate its efficiency in discerning applicant quality compared to narrative letters of recommendation (NLORs). STUDY DESIGN Cross-sectional cohort study. METHODS NLORs for 48 otolaryngology residency applicants interviewed from the 2016 application cycle were identified and mined for descriptive terms to generate a word cloud, referred to as a VLOR. Eight individuals reviewed and rated a total of 187 blinded NLORs and 48 VLORs on a four-point scale (negative to exceptional). Median VLOR and NLOR scores and the time to review for each candidate were compared using the Wilcoxon signed rank test. RESULTS It took significantly more time to review the NLORs in comparison to the VLORs (67 seconds, interquartile range [IQR]: 41-98 seconds vs. 17 seconds, IQR: 11-26 seconds, P