{"pk":28506,"title":"Controlling Attention To Solve Working Memory Tasks Using aMemory-Augmented Neural Network","subtitle":null,"abstract":"We introduce a memory-augmented neural network, calledDifferentiable Working Memory (DWM), that captures somekey aspects of attention in working memory. We tested DWMon a suite of psychology inspired tasks, where the model had todevelop a strategy only by processing sequences of inputs anddesired outputs. Thanks to novel attention control mechanismscalled bookmarks, the model was able to rapidly learn a goodstrategy—generalizing to sequence lengths even two orders ofmagnitude larger than that used for training—allowing it to re-tain, ignore or forget information based on its relevance. Thebehavior of DWM is interpretable and allowed us to analyzeits performance on different tasks. Surprisingly, as the train-ing progressed, we observed that in some cases the model wasable to discover more than one successful strategy, possiblyinvolving sophisticated use of memory and attention.","language":"eng","license":{"name":"","short_name":"","text":null,"url":""},"keywords":[],"section":"Papers with Oral Presentations","is_remote":true,"remote_url":"https://escholarship.org/uc/item/71x6k26t","frozenauthors":[{"first_name":"T.","middle_name":"S.","last_name":"Jayram","name_suffix":"","institution":"IBM Research AI","department":""},{"first_name":"Younes","middle_name":"","last_name":"Bouhadjar","name_suffix":"","institution":"IBM Research AI","department":""},{"first_name":"Tomasz","middle_name":"","last_name":"Kornuta","name_suffix":"","institution":"IBM Research AI","department":""},{"first_name":"Ryan","middle_name":"L.","last_name":"McAvoy","name_suffix":"","institution":"IBM Research AI","department":""},{"first_name":"Alexis","middle_name":"","last_name":"Asseman","name_suffix":"","institution":"IBM Research AI","department":""},{"first_name":"Ahmet","middle_name":"S.","last_name":"Ozcan","name_suffix":"","institution":"IBM Research AI","department":""}],"date_submitted":null,"date_accepted":null,"date_published":"2019-01-01T18:00:00Z","render_galley":null,"galleys":[{"label":"PDF","type":"pdf","path":"https://journalpub.escholarship.org/cognitivesciencesociety/article/28506/galley/18377/download/"}]}