Sentence Selection for Cloze Item Creation: A Standardized Task and Preliminary Results

Cloze items are commonly used for both assessing learning and as a learning activity. This paper investigates the selection of sentences for cloze item creation by comparing methods ranging from simple heuristics to deep learning summarization models. An evaluation using human-generated cloze items from three different science texts indicates that simple heuristics substantially outperform summarization models, including state-of-the-art deep learning models. These results suggest that sentence selection for cloze item generation should be considered a distinct task from summarization and that continued advances on this task will require large datasets of human-generated cloze items. [This paper was published in: "Joint Proceedings of the Workshops at the 14th International Conference on Educational Data Mining," Vol. 3051, LDI-6, edited by T. W. Price and S. San Pedro, CEUR-WS.org, 2021.]