Paper: Refining Generative Language Models using Discriminative Learning

ACL ID D08-1006
Title Refining Generative Language Models using Discriminative Learning
Venue Conference on Empirical Methods in Natural Language Processing
Session Main Conference
Year 2008
Authors

We propose a new approach to language mod- eling which utilizes discriminative learning methods. Our approach is an iterative one: starting with an initial language model, in each iteration we generate 'false' sentences from the current model, and then train a clas- sifier to discriminate between them and sen- tences from the training corpus. To the extent that this succeeds, the classifier is incorpo- rated into the model by lowering the probabil- ity of sentences classified as false, and the process is repeated. We demonstrate the effec- tiveness of this approach on a natural lan- guage corpus and show it provides an 11.4% improvement in perplexity over a modified kneser-ney smoothed trigram.