{"pk":24479,"title":"The Task Task: Creative problem generation in humans and language models","subtitle":null,"abstract":"Machine creativity is on the rise. Recent studies find that large language models achieve human performance on common psychological tests of creativity, which often pose a given problem and ask for novel or unusual solutions. But can AI go beyond producing solutions for given problems, to creatively propose new problems? We present the Task Task, a novel test that asks participants to come up with creative problems. In this test, we assess the ability of humans and GPT-4 to design challenge tasks for a game show. We evaluated proposed tasks using crowdsourced subjective creativity ratings, as well as computational measures of linguistic complexity and semantic content. We found that GPT-4 achieves similar scores as humans on creativity, originality, and judgments of how fun or difficult the tasks are. However, model-generated output tends to be shorter and connect more semantically distant concepts. We discuss implications and future directions for the psychology of creativity.","language":"eng","license":{"name":"","short_name":"","text":null,"url":""},"keywords":[{"word":"Artificial Intelligence; Psychology; Creativity; Natural Language Processing; Reasoning; Computer-based experiment; Large Language Models"}],"section":"Papers with Poster Presentation","is_remote":true,"remote_url":"https://escholarship.org/uc/item/8nn3d4zw","frozenauthors":[{"first_name":"Junyi","middle_name":"","last_name":"Chu","name_suffix":"","institution":"Harvard University","department":""},{"first_name":"Jennifer","middle_name":"","last_name":"Hu","name_suffix":"","institution":"Harvard University","department":""},{"first_name":"Tomer D.","middle_name":"","last_name":"Ullman","name_suffix":"","institution":"Harvard University","department":""}],"date_submitted":null,"date_accepted":null,"date_published":"2024-01-01T18:00:00Z","render_galley":{"label":"PDF","type":"pdf","path":"https://journalpub.escholarship.org/cognitivesciencesociety/article/24479/galley/21657/download/"},"galleys":[{"label":"PDF","type":"pdf","path":"https://journalpub.escholarship.org/cognitivesciencesociety/article/24479/galley/14076/download/"},{"label":"PDF","type":"pdf","path":"https://journalpub.escholarship.org/cognitivesciencesociety/article/24479/galley/21657/download/"}]}