Physics on autopilot: exploring the use of an AI assistant for independent problem-solving practice

This study investigates the efficacy of large language model (LLM)-powered chatbots in guiding physics problem-solving, examining whether they can effectively supplement teacher-led learning. A customised chatbot was developed leveraging ChatGPT to provide step-by-step assistance through a structured problem-solving algorithm. Its impact was evaluated via an experimental study with 12th-grade physics students (N = 24) randomly assigned to a teacher-guided or chatbot-guided group for problem-solving practice. A Mann-Whitney U test revealed no significant differences in problem-solving competency between conditions. Qualitative analysis of conversational logs indicates the chatbot successfully emulated key teacher scaffolding behaviours. Our findings suggest AI tutors can deliver personalised, interactive support akin to human teachers, offering viable supplements to augment physics learning. Further research should explore optimising LLM training, human-chatbot balances, and impacts across diverse educational settings.