Paper: A Comparison Of Algorithms For Maximum Entropy Parameter Estimation

ACL ID W02-2018
Title A Comparison Of Algorithms For Maximum Entropy Parameter Estimation
Venue International Conference on Computational Natural Language Learning
Session Main Conference
Year 2002
Authors
  • Robert Malouf (University of Groningen, Groningen The Netherlands)

Conditional maximum entropy (ME) models pro- vide a general purpose machine learning technique which has been successfully applied to fields as diverse as computer vision and econometrics, and which is used for a wide variety of classification problems in natural language processing. However, the flexibility of ME models is not without cost. While parameter estimation for ME models is con- ceptually straightforward, in practice ME models for typical natural language tasks are very large, and may well contain many thousands of free parame- ters. In this paper, we consider a number of algo- rithms for estimating the parameters of ME mod- els, including iterative scaling, gradient ascent, con- jugate gradient, and variable metric methods. Sur- prisingly, the standardly used iterative scaling ...