Paper: An Empirical Study Of Smoothing Techniques For Language Modeling

ACL ID P96-1041
Title An Empirical Study Of Smoothing Techniques For Language Modeling
Venue Annual Meeting of the Association of Computational Linguistics
Session Main Conference
Year 1996
Authors

We present an extensive empirical com- parison of several smoothing techniques in the domain of language modeling, includ- ing those described by Jelinek and Mer- cer (1980), Katz (1987), and Church and Gale (1991). We investigate for the first time how factors such as training data size, corpus (e.g. , Brown versus Wall Street Journal), and n-gram order (bigram versus trigram) affect the relative performance of these methods, which we measure through the cross-entropy of test data. In addition, we introduce two novel smoothing tech- niques, one a variation of Jelinek-Mercer smoothing and one a very simple linear in- terpolation technique, both of which out- perform existing methods.