Paper: Aggregate And Mixed-Order Markov Models For Statistical Language Processing

ACL ID W97-0309
Title Aggregate And Mixed-Order Markov Models For Statistical Language Processing
Venue Conference on Empirical Methods in Natural Language Processing
Session Main Conference
Year 1997
Authors

We consider the use of language models whose size and accuracy are intermedi- ate between different order n-gram models. Two types of models are studied in partic- ular. Aggregate Markov models are class- based bigram models in which the map- ping from words to classes is probabilis- tic. Mixed-order Markov models combine bigram models whose predictions are con- ditioned on different words. Both types of models are trained by Expectation- Maximization (EM) algorithms for maxi- mum likelihood estimation. We examine smoothing procedures in which these mod- els are interposed between different order n-grams. This is found to significantly re- duce the perplexity of unseen word combi- nations.