Conditional maximum entropy (ME) models pro- vide a general purpose machine learning technique which has been successfully applied to fields as diverse as computer vision and econometrics, and which is used for a wide variety of classification problems in natural language processing. However, the flexibility of ME models is not without cost. While parameter estimation for ME models is con- ceptually straightforward, in practice ME models for typical natural language tasks are very large, and may well contain many thousands of free parame- ters. In this paper, we consider a number of algo- rithms for estimating the parameters of ME mod- els, including iterative scaling, gradient ascent, con- jugate gradient, and variable metric methods. Sur- prisingly, the standardly used iterative scaling ...

Click Here to View the Paper In XML Format