Paper: Deriving lexical and syntactic expectation-based measures for psycholinguistic modeling via incremental top-down parsing

ACL ID D09-1034
Title Deriving lexical and syntactic expectation-based measures for psycholinguistic modeling via incremental top-down parsing
Venue Conference on Empirical Methods in Natural Language Processing
Session Main Conference
Year 2009
Authors

A number of recent publications have made use of the incremental output of stochastic parsers to derive measures of high utility for psycholinguistic modeling, following the work of Hale (2001; 2003; 2006). In this paper, we present novel methods for calculating separate lexical and syntactic surprisal measures from a single incremental parser using a lexical- ized PCFG. We also present an approx- imation to entropy measures that would otherwise be intractable to calculate for a grammar of that size. Empirical results demonstrate the utility of our methods in predicting human reading times.