1. Using Latent Semantic Analysis to Score Short Answer Constructed Responses: Automated Scoring of the Consequences Test
- Author
-
Robert N. Kilcullen, Sharon Ardison, James D. A. Parker, Noelle LaVoie, and Peter J. Legree
- Subjects
Computer science ,business.industry ,Latent semantic analysis ,Applied Mathematics ,05 social sciences ,Short answer ,050401 social sciences methods ,050301 education ,computer.software_genre ,Semantics ,Article ,Education ,Test (assessment) ,Correlation ,Range (mathematics) ,0504 sociology ,Developmental and Educational Psychology ,Artificial intelligence ,Creative thinking ,business ,0503 education ,computer ,Divergent thinking ,Applied Psychology ,Natural language processing - Abstract
Automated scoring based on Latent Semantic Analysis (LSA) has been successfully used to score essays and constrained short answer responses. Scoring tests that capture open-ended, short answer responses poses some challenges for machine learning approaches. We used LSA techniques to score short answer responses to the Consequences Test, a measure of creativity and divergent thinking that encourages a wide range of potential responses. Analyses demonstrated that the LSA scores were highly correlated with conventional Consequence Test scores, reaching a correlation of .94 with human raters and were moderately correlated with performance criteria. This approach to scoring short answer constructed responses solves many practical problems including the time for humans to rate open-ended responses and the difficulty in achieving reliable scoring.
- Published
- 2019