Paper: Parser Evaluation Using Derivation Trees: A Complement to evalb

ACL ID P14-2109
Title Parser Evaluation Using Derivation Trees: A Complement to evalb
Venue Annual Meeting of the Association of Computational Linguistics
Session Main Conference
Year 2014
Authors

This paper introduces a new technique for phrase-structure parser analysis, catego- rizing possible treebank structures by inte- grating regular expressions into derivation trees. We analyze the performance of the Berkeley parser on OntoNotes WSJ and the English Web Treebank. This provides some insight into the evalb scores, and the problem of domain adaptation with the web data. We also analyze a ?test-on- train? dataset, showing a wide variance in how the parser is generalizing from differ- ent structures in the training material.