We investigate the empirical behavior of n- gram discounts within and across domains. When a language model is trained and evalu- ated on two corpora from exactly the same do- main, discounts are roughly constant, match- ing the assumptions of modified Kneser-Ney LMs. However, when training and test corpora diverge, the empirical discount grows essen- tially as a linear function of the n-gram count. We adapt a Kneser-Ney language model to incorporate such growing discounts, result- ing in perplexity improvements over modified Kneser-Ney and Jelinek-Mercer baselines.

Click Here to View the Paper In XML Format