{"pk":49640,"title":"Eliciting the Priors of Large Language Models using Iterated In-Context Learning","subtitle":null,"abstract":"As Large Language Models (LLMs) are increasingly deployed in real-world settings, understanding the knowledge they implicitly use when making decisions is critical. One way to capture this knowledge is in the form of Bayesian prior distributions. We develop a prompt-based workflow for eliciting prior distributions from LLMs. Our approach is based on iterated learning, a method that has been used to explore implicit knowledge in human decision-makers in which successive inferences are chained together to converge to the prior distribution. We validated our method in settings where iterated learning has previously been used to estimate the priors of  human participants -- causal learning, proportion estimation, and predicting everyday quantities. We found that priors elicited from GPT-4 qualitatively align with human priors in these settings. We then used the same method to elicit priors from GPT-4 for a variety of speculative events, such as the timing of the development of superhuman AI.","language":"eng","license":{"name":"","short_name":"","text":null,"url":""},"keywords":[{"word":"Artificial Intelligence; Computer Science; Psychology; Behavioral Science; Concepts and categories; Decision making; Intelligent agents; Machine learning; Natural Language Processing; Representation; "}],"section":"Papers with Poster Presentation","is_remote":true,"remote_url":"https://escholarship.org/uc/item/60d9g74t","frozenauthors":[{"first_name":"Jian-Qiao","middle_name":"","last_name":"Zhu","name_suffix":"","institution":"Princeton University","department":""},{"first_name":"Tom","middle_name":"","last_name":"Griffiths","name_suffix":"","institution":"Princeton University","department":""}],"date_submitted":null,"date_accepted":null,"date_published":"2025-01-01T19:00:00+01:00","render_galley":null,"galleys":[{"label":"PDF","type":"pdf","path":"https://journalpub.escholarship.org/cognitivesciencesociety/article/49640/galley/37602/download/"}]}