{"pk":50266,"title":"Temporal-Difference Learning in Uncertain Choice: A Reinforcement Learning-Diffusion Decision Model of Two-Stage Decision-Making","subtitle":null,"abstract":"Behavioral adaptation in probabilistic environments requires learning through trial and error. While reinforcement learning (RL) models can describe the temporal development of preferences through error-driven learning, the diffusion decision model (DDM) allow for the mapping of state preferences on single response times. We present a Bayesian hierarchical RL-DDM integrating temporal-difference (TD) learning. Our implementation incorporates variants of TD learning, including SARSA, Q-Learning, and Actor-Critic models. We tested the model with data from N = 59 participants in a two-stage decision-making task. Participants exhibited learning over time, becoming both more accurate and faster. They also reflected a difficulty effect, with faster and more accurate responses for easier choices, as reflected by greater subjective value differences between available options. Model comparison demonstrated that the RL-DDM provided a better fit compared to standalone RL or DDM models. Notably, the RL-DDM captured both the temporal dynamics of learning and the difficulty effect in decision-making.","language":"eng","license":{"name":"","short_name":"","text":null,"url":""},"keywords":[{"word":"Psychology; Decision making; Learning; Bayesian modeling; Computational Modeling; Computer-based experiment; Mathematical modeling"}],"section":"Member Abstracts with Poster Presentation","is_remote":true,"remote_url":"https://escholarship.org/uc/item/4049p79k","frozenauthors":[{"first_name":"Nicola","middle_name":"","last_name":"Schneider","name_suffix":"","institution":"Heidelberg University","department":""},{"first_name":"Andreas","middle_name":"","last_name":"Voss","name_suffix":"","institution":"Heidelberg University","department":""}],"date_submitted":null,"date_accepted":null,"date_published":"2025-01-01T10:00:00-08:00","render_galley":null,"galleys":[{"label":"PDF","type":"pdf","path":"https://journalpub.escholarship.org/cognitivesciencesociety/article/50266/galley/38228/download/"}]}