{"pk":29407,"title":"Neural Language Models Capture Some, But Not All, Agreement AttractionEffects","subtitle":null,"abstract":"The number of the subject in English must match the num-ber of the corresponding verb (dog runs but dogs run). Yetin real-time language production and comprehension, speak-ers often mistakenly compute agreement between the verb anda grammatically irrelevant non-subject noun phrase instead.This phenomenon, referred to as agreement attraction, is mod-ulated by a wide range of factors; any complete computationalmodel of grammatical planning and comprehension would beexpected to derive this rich empirical picture. Recent develop-ments in Natural Language Processing have shown that neuralnetworks trained only on word-prediction over large corporaare capable of capturing subject-verb agreement dependen-cies to a significant extent, but with occasional errors. In thispaper, we evaluate the potential of such neural word predic-tion models as a foundation for a cognitive model of real-timegrammatical processing. We use LSTMs, a common sequenceprediction model used to model language, to simulate six ex-periments taken from the agreement attraction literature. TheLSTMs captured the critical human behavior in three out of thesix experiments, indicating that (1) some agreement attractionphenomena can be captured by a generic sequence process-ing model, but (2) capturing the other phenomena may requiremodels with more language-specific mechanisms.","language":"eng","license":{"name":"","short_name":"","text":null,"url":""},"keywords":[{"word":"psycholinguistics; computational modeling;agreement attraction; neural language models;"}],"section":"Neural Networks","is_remote":true,"remote_url":"https://escholarship.org/uc/item/7782c9vz","frozenauthors":[{"first_name":"Suhas","middle_name":"","last_name":"Arehalli","name_suffix":"","institution":"Johns Hopkins University","department":""},{"first_name":"Tal","middle_name":"","last_name":"Linzen","name_suffix":"","institution":"Johns Hopkins University","department":""}],"date_submitted":null,"date_accepted":null,"date_published":"2020-01-01T18:00:00Z","render_galley":null,"galleys":[{"label":"PDF","type":"pdf","path":"https://journalpub.escholarship.org/cognitivesciencesociety/article/29407/galley/19267/download/"}]}