Paper: Bootstrapping Feature-Rich Dependency Parsers with Entropic Priors

ACL ID D07-1070
Title Bootstrapping Feature-Rich Dependency Parsers with Entropic Priors
Venue Conference on Empirical Methods in Natural Language Processing
Session Main Conference
Year 2007
Authors

One may need to build a statistical parser for a new language, using only a very small labeled treebank together with raw text. We argue that bootstrapping a parser is most promising when the model uses a rich set of redundant features, as in re- cent models for scoring dependency parses (McDonald et al. , 2005). Drawing on Abney’s (2004) analysis of the Yarowsky algorithm, we perform bootstrapping by entropy regulariza- tion: we maximize a linear combination of conditional likeli- hood on labeled data and confidence (negative R´enyi entropy) on unlabeled data. In initial experiments, this surpassed EM for training a simple feature-poor generative model, and also improved the performance of a feature-rich, conditionally esti- mated model where EM could not easily have been applied. For ...