Credibility and altered communication styles of AI graders in the classroom.

Background: Education is often the primary arena for exploring and integrating new technologies. AI and human‐machine communication (HMC) are prevalent in the classroom, yet we are still learning how student perceptions of these tools will impact education. Objectives: We sought to understand student perceptions of credibility related to written feedback attributed to a human or an AI grader (Study One). We also investigated how corrective messages containing verbal immediacy and social support influenced student perceptions of an AI grader's credibility based on feedback in an evaluated essay (Study Two). Methods: We used an online experimental design to assess the perceived credibility of a grader. In Study One, we randomly assigned students (N = 155) to a condition that contained a paragraph they were told was evaluated by a human or an AI grader. In Study Two (N = 222), we investigated ways of increasing perceptions of an AI grader's credibility by writing messages with higher/lower levels of immediacy and social support. Results: In Study One, the students rated both the human and AI grader as credible (yet rated the AI grader lower on goodwill). The data suggest that students in Study Two attributed more goodwill (i.e., caring) to the AI grader when the feedback included more verbal immediacy. Conclusions: Our results highlight the importance of student perceptions and communication styles when integrating technology into education. The two studies imply that students viewed the human and AI graders as competent, caring, and trustworthy, specifically when feedback included more immediacy cues. Lay Description: What is currently known about this topic: An instructor's use of verbal and non‐verbal behaviours is a crucial component of establishing credibility within a student‐instructor relationship.Non‐human actors (artificial intelligence and social robots) are perceived as credible when they use appropriate verbal and non‐verbal messages.Education plays a central role in exploring and integrating new technologies, but initial impressions and perceived utility impact acceptance and adoption. What does this paper add: Students perceived both human and machine graders as credible based on measures of competence, goodwill, and trustworthiness.This is the first study that sought to alter perceptions of a machine grader's credibility by manipulating verbal and non‐verbal immediacy cues.Results suggest that a machine grader's communication style, especially verbal immediacy, would influence student engagement. Implications for practice/or policy: This study offers initial insights into using machine grading as a supplementary engagement tool.Communication style is a key factor to consider when designing courses that include automated scoring systems or AI graders. [ABSTRACT FROM AUTHOR]