Paper: Using Perfect Sampling In Parameter Estimation Of A Whole Sentence Maximum Entropy Language Model

ACL ID W00-0714
Title Using Perfect Sampling In Parameter Estimation Of A Whole Sentence Maximum Entropy Language Model
Venue International Conference on Computational Natural Language Learning
Session Main Conference
Year 2000
Authors

The Maximum Entropy principle (ME) is an ap- propriate framework for combining information of a diverse nature from several sources into the same language model. In order to incorporate long-distance information into the ME frame- work in a language model, a Whole Sentence Maximum Entropy Language Model (WSME) could be used. Until now MonteCarlo Markov Chains (MCMC) sampling techniques has been used to estimate the paramenters of the WSME model. In this paper, we propose the applica- tion of another sampling technique: the Perfect Sampling (PS). The experiment has shown a re- duction of 30% in the perplexity of the WSME model over the trigram model and a reduc- tion of 2% over the WSME model trained with MCMC.