{"pk":27760,"title":"Evidence for hierarchically-structured reinforcement learning in humans","subtitle":null,"abstract":"Flexibly adapting behavior to different contexts is a critical\ncomponent of human intelligence. It requires knowledge to\nbe structured as coherent, context-dependent action rules, or\ntask-sets (TS). Nevertheless, inferring optimal TS is compu-\ntationally complex. This paper tests the key predictions of a\nneurally-inspired model that employs hierarchically-structured\nreinforcement learning (RL) to approximate optimal inference.\nThe model proposes that RL acts at two levels of abstrac-\ntion: a high-level RL process learns context-TS values, which\nguide TS selection based on context; a low-level process learns\nstimulus-actions values within TS, which guide action selec-\ntion in response to stimuli. In our novel task paradigm, we\nfound evidence that participants indeed learned values at both\nlevels: not only stimulus-action values, but also context-TS\nvalues affected learning and TS reactivation, and TS values\nalone determined TS generalization. This supports the claim\nof two RL processes, and their importance in structuring our\ninteractions with the world.","language":"eng","license":{"name":"","short_name":"","text":null,"url":""},"keywords":[{"word":"Reinforcement Learning"},{"word":"Structure learning"},{"word":"Hierarchical representation"},{"word":"Task sets"}],"section":"Publication-based-Talks","is_remote":true,"remote_url":"https://escholarship.org/uc/item/3wx3881m","frozenauthors":[{"first_name":"Maria","middle_name":"K","last_name":"Eckstein","name_suffix":"","institution":"University of California, Berkley","department":""},{"first_name":"Anne","middle_name":"GE","last_name":"Collins","name_suffix":"","institution":"University of California, Berkley","department":""}],"date_submitted":null,"date_accepted":null,"date_published":"2018-01-01T18:00:00Z","render_galley":null,"galleys":[{"label":"PDF","type":"pdf","path":"https://journalpub.escholarship.org/cognitivesciencesociety/article/27760/galley/17400/download/"}]}