Opening the Black Box of College Counseling. CEPA Working Paper No. 20-03

Although many programs remotely disseminate information to students about the college application process, there is little evidence as to how students experience these programs. This paper examines a large-scale remote counseling program in which college counselors initiated interactions with 15,000 high school seniors via text message to support them through the college application process. Given the passive nature of text messaging, not all of the counselors' prompts elicited similar responses from students. I use text-as-data methods (combining qualitative coding and supervised machine learning) to measure which interactions lead to productive engagement between counselors and students, and which do not. I show that interactions about financial aid offers and financial aid applications are much more likely to generate productive engagement than interactions about college lists. This finding may help to explain why recent remote counseling interventions that have sought to influence students' college lists have been ineffective.