{"pk":24537,"title":"Estimating human color-concept associations from multimodal language models","subtitle":null,"abstract":"People's color-concept associations influence many processes underlying visual cognition from object recognition to information visualization interpretation. Thus, a key goal in cognitive science is developing efficient methods for estimating color-concept association distributions over color space to model these processes. Here, we investigated the ability of GPT-4, a multimodal large language model, to estimate human-like color-concept associations. We collected human association ratings between 70 concepts spanning abstractness and 71 colors spanning perceptual color space and compared these ratings to analogous ratings from GPT-4, when it was given concepts as words and colors as hexadecimal codes. GPT-4 ratings were correlated with human ratings, comparably to state-of-the-art image-based methods. Variation in human-GPT rating correlations across concepts was predicted by concept abstractness, but this effect was superseded by specificity (peakiness; inverse entropy) of color-concept association distributions. Our results highlight the viability of using model-generated color-concept association ratings to better understand human color semantics.","language":"eng","license":{"name":"","short_name":"","text":null,"url":""},"keywords":[{"word":"Artificial Intelligence; Concepts and categories; Perception; Vision; Large Language Models"}],"section":"Abstracts","is_remote":true,"remote_url":"https://escholarship.org/uc/item/6dh916d5","frozenauthors":[{"first_name":"Kushin","middle_name":"","last_name":"Mukherjee","name_suffix":"","institution":"University of Wisconsin-Madison","department":""},{"first_name":"Timothy","middle_name":"","last_name":"Rogers","name_suffix":"","institution":"UW-Madison","department":""},{"first_name":"Karen","middle_name":"","last_name":"Schloss","name_suffix":"","institution":"University of Wisconsin - Madison","department":""}],"date_submitted":null,"date_accepted":null,"date_published":"2024-01-01T18:00:00Z","render_galley":{"label":"PDF","type":"pdf","path":"https://journalpub.escholarship.org/cognitivesciencesociety/article/24537/galley/21139/download/"},"galleys":[{"label":"PDF","type":"pdf","path":"https://journalpub.escholarship.org/cognitivesciencesociety/article/24537/galley/14134/download/"},{"label":"PDF","type":"pdf","path":"https://journalpub.escholarship.org/cognitivesciencesociety/article/24537/galley/21139/download/"}]}