Ramteja Sajja, Yusuf Sermet, Ibrahim Demir

This study introduces an end-to-end framework for deploying conversational AI-enabled educational assistants, focusing on personalized support for students across diverse subject areas, including Business, Culture, Environmental Sciences, History, Politics, and Science, as outlined in our evaluation framework. The system leverages advanced conversational AI technologies to provide targeted, course-specific learning experiences by facilitating access to complex data and integrating seamlessly with Learning Management Systems (LMS) like Canvas. Key metrics—information retrieval accuracy, question-answering accuracy, and hallucination accuracy—were selected to rigorously evaluate the system’s ability to retrieve relevant contexts, generate accurate responses, and identify unanswerable questions. Additionally, the Educational AI Hub agents utilize innovative document parsing methods, such as the Nougat technique, to interpret content accurately, enabling adaptable academic support tailored to individual learning needs and extending to quantitative fields through code execution capabilities. This study also emphasizes the importance of accessibility, inclusivity, and user privacy. The results showcase the potential for enhanced engagement and improved understanding of environmental concepts and software tools, demonstrating the significant impact of conversational AI in educational settings, especially in disciplines involving complex data interactions. A case study, presented at the 12th International Congress on Environmental Modelling and Software, illustrates the Educational AI Hub's effectiveness in enhancing student engagement and delivering personalized learning experiences in environmental sciences education.