{"pk":26943,"title":"Evaluating Vector-Space Models of Word Representation, or,The Unreasonable Effectiveness of Counting Words Near Other Words","subtitle":null,"abstract":"Vector-space models of semantics represent words ascontinuously-valued vectors and measure similarity based onthe distance or angle between those vectors. Such representa-tions have become increasingly popular due to the recent de-velopment of methods that allow them to be efficiently esti-mated from very large amounts of data. However, the ideaof relating similarity to distance in a spatial representationhas been criticized by cognitive scientists, as human similar-ity judgments have many properties that are inconsistent withthe geometric constraints that a distance metric must obey. Weshow that two popular vector-space models, Word2Vec andGloVe, are unable to capture certain critical aspects of humanword association data as a consequence of these constraints.However, a probabilistic topic model estimated from a rela-tively small curated corpus qualitatively reproduces the asym-metric patterns seen in the human data. We also demonstratethat a simple co-occurrence frequency performs similarly toreduced-dimensionality vector-space models on medium-sizecorpora, at least for relatively frequent words.","language":"eng","license":{"name":"","short_name":"","text":null,"url":""},"keywords":[{"word":"word representations; vector-space models; wordassociations"}],"section":"Talks: Papers","is_remote":true,"remote_url":"https://escholarship.org/uc/item/1kh9p4gj","frozenauthors":[{"first_name":"Aida","middle_name":"","last_name":"Nematzadeh","name_suffix":"","institution":"University of California, Berkeley","department":""},{"first_name":"Stephan","middle_name":"C.","last_name":"Meylan","name_suffix":"","institution":"University of California, Berkeley","department":""},{"first_name":"Thomas","middle_name":"L.","last_name":"Griffiths","name_suffix":"","institution":"University of California, Berkeley","department":""}],"date_submitted":null,"date_accepted":null,"date_published":"2017-01-01T18:00:00Z","render_galley":null,"galleys":[{"label":"PDF","type":"pdf","path":"https://journalpub.escholarship.org/cognitivesciencesociety/article/26943/galley/16579/download/"}]}