{"pk":29470,"title":"Universal linguistic inductive biases via meta-learning","subtitle":null,"abstract":"How do learners acquire languages from the limited data avail-able to them? This process must involve some inductivebiases—factors that affect how a learner generalizes—but it isunclear which inductive biases can explain observed patternsin language acquisition. To facilitate computational model-ing aimed at addressing this question, we introduce a frame-work for giving particular linguistic inductive biases to a neu-ral network model; such a model can then be used to em-pirically explore the effects of those inductive biases. Thisframework disentangles universal inductive biases, which areencoded in the initial values of a neural network’s param-eters, from non-universal factors, which the neural networkmust learn from data in a given language. The initial statethat encodes the inductive biases is found with meta-learning,a technique through which a model discovers how to acquirenew languages more easily via exposure to many possible lan-guages. By controlling the properties of the languages that areused during meta-learning, we can control the inductive biasesthat meta-learning imparts. We demonstrate this frameworkwith a case study based on syllable structure. First, we specifythe inductive biases that we intend to give our model, and thenwe translate those inductive biases into a space of languagesfrom which a model can meta-learn. Finally, using existinganalysis techniques, we verify that our approach has impartedthe linguistic inductive biases that it was intended to impart.","language":"eng","license":{"name":"","short_name":"","text":null,"url":""},"keywords":[{"word":"Meta-learning"},{"word":"inductive bias"},{"word":"language univer-sals"},{"word":"syllable structure typology"},{"word":"Neural Networks"}],"section":"Linguistics","is_remote":true,"remote_url":"https://escholarship.org/uc/item/93m7w30j","frozenauthors":[{"first_name":"R.","middle_name":"Thomas","last_name":"McCoy","name_suffix":"","institution":"Johns Hopkins University","department":""},{"first_name":"Erin","middle_name":"","last_name":"Grant","name_suffix":"","institution":"University of California, Berkeley","department":""},{"first_name":"Paul","middle_name":"","last_name":"Smolensky","name_suffix":"","institution":"Microsoft Research AI , Johns Hopkins University","department":""},{"first_name":"Thomas","middle_name":"L.","last_name":"Griffiths","name_suffix":"","institution":"Princeton University","department":""},{"first_name":"Tal","middle_name":"","last_name":"Linzen","name_suffix":"","institution":"Johns Hopkins University","department":""}],"date_submitted":null,"date_accepted":null,"date_published":"2020-01-01T18:00:00Z","render_galley":null,"galleys":[{"label":"PDF","type":"pdf","path":"https://journalpub.escholarship.org/cognitivesciencesociety/article/29470/galley/19330/download/"}]}