Paper: A Comparative Study of Parameter Estimation Methods for Statistical Natural Language Processing

ACL ID P07-1104
Title A Comparative Study of Parameter Estimation Methods for Statistical Natural Language Processing
Venue Annual Meeting of the Association of Computational Linguistics
Session Main Conference
Year 2007
Authors

This paper presents a comparative study of five parameter estimation algorithms on four NLP tasks. Three of the five algorithms are well-known in the computational linguistics community: Maximum Entropy (ME) estima- tion with L2 regularization, the Averaged Perceptron (AP), and Boosting. We also in- vestigate ME estimation with L1 regularization using a novel optimization algorithm, and BLasso, which is a version of Boosting with Lasso (L1) regularization. We first investigate all of our estimators on two re-ranking tasks: a parse selection task and a language model (LM) adaptation task. Then we apply the best of these estimators to two additional tasks involving conditional sequence models: a Conditional Markov Model (CMM) for part of speech tagging and a Conditional Random Field (CRF) fo...