1. AUTALIC: A Dataset for Anti-AUTistic Ableist Language In Context
- Author
-
Rizvi, Naba, Strickland, Harper, Gitelman, Daniel, Cooper, Tristan, Morales-Flores, Alexis, Golden, Michael, Kallepalli, Aekta, Alurkar, Akshat, Owens, Haaset, Ahmedi, Saleha, Khirwadkar, Isha, Munyaka, Imani, and Ousidhoum, Nedjma
- Subjects
Computer Science - Computation and Language ,Computer Science - Artificial Intelligence - Abstract
As our understanding of autism and ableism continues to increase, so does our understanding of ableist language towards autistic people. Such language poses a significant challenge in NLP research due to its subtle and context-dependent nature. Yet, detecting anti-autistic ableist language remains underexplored, with existing NLP tools often failing to capture its nuanced expressions. We present AUTALIC, the first benchmark dataset dedicated to the detection of anti-autistic ableist language in context, addressing a significant gap in the field. The dataset comprises 2,400 autism-related sentences collected from Reddit, accompanied by surrounding context, and is annotated by trained experts with backgrounds in neurodiversity. Our comprehensive evaluation reveals that current language models, including state-of-the-art LLMs, struggle to reliably identify anti-autistic ableism and align with human judgments, underscoring their limitations in this domain. We publicly release AUTALIC along with the individual annotations which serve as a valuable resource to researchers working on ableism, neurodiversity, and also studying disagreements in annotation tasks. This dataset serves as a crucial step towards developing more inclusive and context-aware NLP systems that better reflect diverse perspectives., Comment: 9 pages, 5 figures, 7 tables
- Published
- 2024