iLeap: A Human-Ai Teaming Based Mobile Language Learning Solution for Dual Language Learners in Early and Special Educations

In this research paper, we present an AR- and AI-based mobile learning tool that provides: 1.) automatic and accurate intelligibility analysis at various levels: letter, word, phrase and sentences, 2.) immediate feedback and multimodal coaching on how to correct pronunciation, and 3.) evidence-based dynamic training curriculum tailored to each individual's learning patterns and needs, e.g., retention of corrected pronunciation and typical pronunciation errors. The use of visible and interactive virtual expert technology capable of intuitive AR-based interactions will greatly increase student's acceptance and retention of a virtual coach. In school or at home, it will readily resemble an expert reading specialist to effectively guide and assist a student in practicing reading and speaking by him-/herself independently, which is particularly important for dual language learners (DLL) whose first language (L1) is not English as many of their parents don't speak English fluently and cannot offer the necessary help. Our human-AI teaming based solution overcomes the shortfall of conventional computer-based language learning tools and serve as a supportive and team-based learning platform that is critical for optimizing the learning outcomes. [For the complete proceedings, see ED601100.]