{"pk":32003,"title":"A Recurrent Network that performs a Conext-Sensitive Prediction Task","subtitle":null,"abstract":"We address the problem of processing a context-sensitive language with a recurrent neural network (RN). So far, the language processing capabilities of RNs have only been investigated for regular and context-free languages. We present an extremely simple RN with only one parameter <i>z</i> for its two hidden nodes that can perform a prediction task on sequences of symbols from the language {(<i>ba^k</i>)<i>^n</i>\" | <i>k</i> &gt;= 0, <i>n</i> &gt; 0}, a language that is context-sensitive but not context-free. The input to the RN consists of any string of the language, one symbol at a time. The network should then, at all times, predict the symbol that should follow. This means that the network must be able to count the number of <i>a</i>'s in the first subsequence and to retain this number for future use. We present a value for the parameter <i>z</i> for which our RN can solve the task for <i>k</i> = 1 up to <i>k</i> = 120. As we do not give any method to find a good value for <i>z</i>, this does not say anything about the learning capabilities of our network. It does, however, show that context-sensitive information (the count of <i>a</i>'s) can be <i>represented</i> by the network; we analyse in detail how this is done. Hence our work shows that, at least from a representational point of view, connectionist architectures can handle more complex formal languages than was previously known.","language":"eng","license":{"name":"","short_name":"","text":null,"url":""},"keywords":[],"section":"Paper Presentations","is_remote":true,"remote_url":"https://escholarship.org/uc/item/7h45x8kx","frozenauthors":[{"first_name":"Mark","middle_name":"","last_name":"Steijvers","name_suffix":"","institution":"Department of Psychology, Indiana University","department":""},{"first_name":"Peter","middle_name":"","last_name":"Grunwald","name_suffix":"","institution":"Department of Algorithmics, CWI","department":""}],"date_submitted":null,"date_accepted":null,"date_published":"1996-01-01T13:00:00-05:00","render_galley":null,"galleys":[{"label":"PDF","type":"pdf","path":"https://journalpub.escholarship.org/cognitivesciencesociety/article/32003/galley/23068/download/"}]}