{"pk":50014,"title":"GPT-4o Lacks Core Features of Theory of Mind","subtitle":null,"abstract":"Do Large Language Models (LLMs) possess a Theory of Mind (ToM)? Research into this question has found that LLMs succeed on a range of benchmark tasks. However, these evaluations do not test for the actual representations posited by ToM: namely, a causal model of mental states and behavior. Here, we use a cognitively-grounded definition of ToM to develop and test a new evaluation framework. Specifically, our approach probes whether LLMs have a coherent, abstract, and consistent model of how mental states cause behavior—regardless of whether that model matches a human-like ToM. We test our evaluation against GPT-4o and find that even though it succeeds in approximating human judgments in a simple ToM paradigm, GPT-4o fails at a logically-equivalent task and exhibits low consistency between its action predictions and corresponding mental state inferences. As such, these findings suggest that GPT-4o's social proficiency is not the result of a ToM.","language":"eng","license":{"name":"","short_name":"","text":null,"url":""},"keywords":[{"word":"Artificial Intelligence; Computer Science; Philosophy; Psychology; Intelligent agents; Machine learning; Natural Language Processing; Social cognition; Theory of Mind; Bayesian modeling; Computational"}],"section":"Abstracts with Poster Presentation","is_remote":true,"remote_url":"https://escholarship.org/uc/item/9h04686p","frozenauthors":[{"first_name":"John","middle_name":"","last_name":"Muchovej","name_suffix":"","institution":"Yale University","department":""},{"first_name":"Amanda","middle_name":"L","last_name":"Royka","name_suffix":"","institution":"Yale University","department":""},{"first_name":"Shane","middle_name":"","last_name":"Lee","name_suffix":"","institution":"Yale University","department":""},{"first_name":"Julian","middle_name":"","last_name":"Jara-Ettinger","name_suffix":"","institution":"Yale University","department":""}],"date_submitted":null,"date_accepted":null,"date_published":"2025-01-01T18:00:00Z","render_galley":null,"galleys":[{"label":"PDF","type":"pdf","path":"https://journalpub.escholarship.org/cognitivesciencesociety/article/50014/galley/37976/download/"}]}