{"pk":27724,"title":"Multimodal Surprisal in the N400 and the Index of Cognitive Activity","subtitle":null,"abstract":"A word’s predictability or surprisal, as determined by cloze\nprobabilities or language models (e.g. Frank, Otten, Galli, &\nVigliocco, 2015) is related to processing effort, in that less ex-\npected words take more effort to process (e.g. Hale, 2001). A\nwords surprisal, however, may also be influenced by the non-\nlinguistic context, such as visual cues: In the visual world\nparadigm (VWP), for example, anticipatory eye movements\nsuggest that comprehenders exploit the scene to predict what\nwill be mentioned next (Altmann & Kamide, 1999). How vi-\nsual context affects word surprisal and processing effort, how-\never, remains unclear. Here, we present evidence that visually-\ndetermined probabilistic expectations for a spoken target word\npredict graded processing effort for that word, in both pupil-\nlometric (ICA) and ERP (N400) measures. These findings\ndemonstrate that the non-linguistic context can immediately\ninfluence both lexical expectations, and surprisal-based pro-\ncessing effort.","language":"eng","license":{"name":"","short_name":"","text":null,"url":""},"keywords":[{"word":"N400"},{"word":"Language Processing"},{"word":"EEG"},{"word":"Index of cognitive activity"},{"word":"prediction"},{"word":"eye-tracking"}],"section":"Publication-based-Talks","is_remote":true,"remote_url":"https://escholarship.org/uc/item/5930b5fd","frozenauthors":[{"first_name":"Christine","middle_name":"S","last_name":"Ankener","name_suffix":"","institution":"Saarland University","department":""},{"first_name":"Heiner","middle_name":"","last_name":"Dernhaus","name_suffix":"","institution":"Saarland University","department":""},{"first_name":"Mathew","middle_name":"W","last_name":"Crocker","name_suffix":"","institution":"Saarland University","department":""},{"first_name":"Maria","middle_name":"","last_name":"Staudte","name_suffix":"","institution":"Saarland University","department":""}],"date_submitted":null,"date_accepted":null,"date_published":"2018-01-01T18:00:00Z","render_galley":null,"galleys":[{"label":"PDF","type":"pdf","path":"https://journalpub.escholarship.org/cognitivesciencesociety/article/27724/galley/17364/download/"}]}