{"pk":28447,"title":"Modifying social dimensions of human faces with ModifAE","subtitle":null,"abstract":"At first glance, humans extract social judgments from faces, in-cluding how trustworthy, attractive, and aggressive they look.These impressions have profound social, economic, and polit-ical consequences, as they subconsciously influence decisionslike voting and criminal sentencing. Therefore, understand-ing human perception of these judgments is important for thesocial sciences. In this work, we present a modifying autoen-coder (ModifAE, pronounced “modify”) that can model andalter these facial impressions. We assemble a face impressiondataset large enough for training a generative model by ap-plying a state-of-the-art (SOTA) impression predictor to facesfrom CelebA. Then, we apply ModifAE to learn generalizablemodifications of these continuous-valued traits in faces (e.g.,make a face look slightly more intelligent or much less aggres-sive). ModifAE can modify face images to create controlledsocial science experimental datasets, and it can reveal datasetbiases by creating direct visualizations of what makes a facesalient in social dimensions. The ModifAE architecture is alsosmaller and faster than SOTA image-to-image translation mod-els, while outperforming SOTA in quantitative evaluations.","language":"eng","license":{"name":"","short_name":"","text":null,"url":""},"keywords":[{"word":"neural networks; generative models; face recogni-tion; social perception; image modification"}],"section":"Papers with Oral Presentations","is_remote":true,"remote_url":"https://escholarship.org/uc/item/0589b5fs","frozenauthors":[{"first_name":"Chad","middle_name":"","last_name":"Atalla","name_suffix":"","institution":"University of California, San Diego","department":""},{"first_name":"Amanda","middle_name":"","last_name":"Song","name_suffix":"","institution":"University of California, San Diego","department":""},{"first_name":"Bartholomew","middle_name":"","last_name":"Tam","name_suffix":"","institution":"University of California, San Diego","department":""},{"first_name":"Asmitha","middle_name":"","last_name":"Rathis","name_suffix":"","institution":"University of California, San Diego","department":""},{"first_name":"Gary","middle_name":"","last_name":"Cottrell","name_suffix":"","institution":"University of California, San Diego","department":""}],"date_submitted":null,"date_accepted":null,"date_published":"2019-01-01T18:00:00Z","render_galley":null,"galleys":[{"label":"PDF","type":"pdf","path":"https://journalpub.escholarship.org/cognitivesciencesociety/article/28447/galley/18318/download/"}]}