{"pk":21693,"title":"Language use is only sparsely compositional: The case of English adjective-noun phrases in humans and large language models","subtitle":null,"abstract":"Compositionality is considered a key hallmark of human\nlanguage. However, most research focuses on item-level compositionality, e.g.,\nto what extent the meanings of phrases are composed of the meanings of their\nsub-parts, rather than on language-level compositionality, which is the degree\nto which possible combinations are utilized in practice during language use.\nHere, we propose a novel way to quantify the degree of language-level\ncompositionality and apply it in the case of English adjective-noun\ncombinations. Using corpus analyses, large language models, and human\nacceptability ratings, we find that (1) English only sparsely utilizes the\ncompositional potential of adjective–noun combinations; and (2) LLMs struggle to\npredict human acceptability judgments of rare combinations. Taken together, our\nfindings shed new light on the role of compositionality in language and\nhighlight a challenging area for further improving LLMs.","language":"eng","license":{"name":"","short_name":"","text":null,"url":""},"keywords":[{"word":"Artificial Intelligence; Computer Science; Linguistics; Language understanding; Natural Language Processing; Semantics; Big data; Computational Modeling; Corpus studies; Large Language Models; Survey"}],"section":"Papers with Poster Presentation","is_remote":true,"remote_url":"https://escholarship.org/uc/item/0qd3662b","frozenauthors":[{"first_name":"Aalok","middle_name":"","last_name":"Sathe","name_suffix":"","institution":"Massachusetts Institute of Technology","department":""},{"first_name":"Evelina","middle_name":"","last_name":"Fedorenko","name_suffix":"","institution":"Massachusetts Institute of Technology","department":""},{"first_name":"Noga","middle_name":"","last_name":"Zaslavsky","name_suffix":"","institution":"UCI","department":""}],"date_submitted":null,"date_accepted":null,"date_published":"2024-01-01T13:00:00-05:00","render_galley":null,"galleys":[{"label":"PDF","type":"pdf","path":"https://journalpub.escholarship.org/cognitivesciencesociety/article/21693/galley/11292/download/"},{"label":"PDF","type":"pdf","path":"https://journalpub.escholarship.org/cognitivesciencesociety/article/21693/galley/22086/download/"}]}