{"pk":26932,"title":"Leveraging Response Consistency within Individuals to Improve Group Accuracyfor Rank-Ordering Problems","subtitle":null,"abstract":"Averaging the estimates of a number of individuals has beenshown to produce an estimate that is generally more accuratethan those of the individuals themselves. Similarly, averagingresponses from a single individual can also lead to a moreaccurate answer. How can we best combine estimates withinand between individuals to create an accurate group estimate?We report empirical results from a general knowledge rank-ordering experiment and demonstrate that individuals thatprovide more consistent answers across repeated elicitationsare also more accurate. We develop a consistency weightingheuristic and show that repeated elicitations within anindividual can be used to improve group accuracy. We alsodevelop a Thurstonian cognitive model which assumes adirect link between the process that explains the accuracy ofan individual and response consistency and show how themodel can infer accurate group answers.","language":"eng","license":{"name":"","short_name":"","text":null,"url":""},"keywords":[{"word":"Bayesian Modeling; Rank Ordering; Knowledge;Recall; Wisdom of Crowds; Within; Expertise; Uncertainty;Coherence; Consistency."}],"section":"Talks: Papers","is_remote":true,"remote_url":"https://escholarship.org/uc/item/40w9476m","frozenauthors":[{"first_name":"Brent","middle_name":"","last_name":"Miller","name_suffix":"","institution":"Vanderbilt University","department":""},{"first_name":"Mark","middle_name":"","last_name":"Steyvers","name_suffix":"","institution":"University of California, Irvine","department":""}],"date_submitted":null,"date_accepted":null,"date_published":"2017-01-01T18:00:00Z","render_galley":null,"galleys":[{"label":"PDF","type":"pdf","path":"https://journalpub.escholarship.org/cognitivesciencesociety/article/26932/galley/16568/download/"}]}