{"pk":49506,"title":"Cognitive Measurement with Generative AI: A Novel Interactive Situational Assessment of Learning Motivation and Strategy Using LLM Multi-Agents","subtitle":null,"abstract":"Assessing learning motivation and strategy (LMS) in specific situations can more accurately reflect students' self-regulation learning ability. However, traditional assessment methods, such as subjective evaluations and self-reports, are time-consuming, burdensome, and not well-suited to the dynamic nature of situational assessments. To address this, we presented the LLM-based agents, which enable intelligent generation of situational tasks and interactive assessment. Specifically, Master defines the theme and storylines, Designers generate situational tasks, Evaluator reviews the content quality, and Interactor controls the interactive assessment with users. The results of a user study with 97 university students demonstrated the reliability and validity of our approach and the significant enhancement of the user experience. The results further clarify the relationship among indicators of LMS. This study provides a novel paradigm and solution for situational assessment of LMS and offers valuable theoretical insights for intervention research targeting related indicators.","language":"eng","license":{"name":"","short_name":"","text":null,"url":""},"keywords":[{"word":"Artificial Intelligence; Education; Human-computer interaction; Intelligent agents; Comparative Studies"}],"section":"Papers with Poster Presentation","is_remote":true,"remote_url":"https://escholarship.org/uc/item/4rn910gr","frozenauthors":[{"first_name":"Yi","middle_name":"","last_name":"Zhang","name_suffix":"","institution":"Jilin University","department":""},{"first_name":"Haotian","middle_name":"","last_name":"Feng","name_suffix":"","institution":"College of Computer Science and Technology","department":""},{"first_name":"Chen","middle_name":"","last_name":"Xue","name_suffix":"","institution":"Jilin University","department":""},{"first_name":"Yatong","middle_name":"","last_name":"Zu","name_suffix":"","institution":"Jilin University","department":""},{"first_name":"Hao","middle_name":"","last_name":"Xu","name_suffix":"","institution":"Jilin University","department":""}],"date_submitted":null,"date_accepted":null,"date_published":"2025-01-01T10:00:00-08:00","render_galley":null,"galleys":[{"label":"PDF","type":"pdf","path":"https://journalpub.escholarship.org/cognitivesciencesociety/article/49506/galley/37468/download/"}]}