Paper: Parser Evaluation over Local and Non-Local Deep Dependencies in a Large Corpus

ACL ID D11-1037
Title Parser Evaluation over Local and Non-Local Deep Dependencies in a Large Corpus
Venue Conference on Empirical Methods in Natural Language Processing
Session Main Conference
Year 2011
Authors

In order to obtain a fine-grained evaluation of parser accuracy over naturally occurring text, we study 100 examples each of ten reason- ably frequent linguistic phenomena, randomly selected from a parsed version of the En- glish Wikipedia. We construct a correspond- ing set of gold-standard target dependencies for these 1000 sentences, operationalize map- pings to these targets from seven state-of-the- art parsers, and evaluate the parsers against this data to measure their level of success in identifying these dependencies.