Lira, Benjamin, Gardner, Margo, Quirk, Abigail, Stone, Cathlyn, Rao, Arjun, Ungar, Lyle, Hutt, Stephen, Hickman, Louis, D'Mello, Sidney K., Duckworth, Angela L., Lira, Benjamin, Gardner, Margo, Quirk, Abigail, Stone, Cathlyn, Rao, Arjun, Ungar, Lyle, Hutt, Stephen, Hickman, Louis, D'Mello, Sidney K., and Duckworth, Angela L.
Personal qualities like prosocial purpose and leadership predict important life outcomes, including college success. Unfortunately, the holistic assessment of personal qualities in college admissions is opaque and resource intensive. Can artificial intelligence (AI) advance the goals of holistic admissions? While cost-effective, AI has been criticized as a "black box" that may inadvertently penalize already disadvantaged subgroups when used in high-stakes settings. Here, we consider an AI approach to assessing personal qualities that aims to overcome these limitations. Research assistants and admissions officers first identified the presence/absence of seven personal qualities in n = 3131 applicant essays describing extracurricular and work experiences. Next, we fine-tuned pretrained language models with these ratings, which successfully reproduced human codes across demographic subgroups. Last, in a national sample (N = 309,594), computer-generated scores collectively demonstrated incremental validity for predicting 6-year college graduation. We discuss challenges and opportunities of AI for assessing personal qualities.