Paper: Language Modeling with Power Low Rank Ensembles

ACL ID D14-1158
Title Language Modeling with Power Low Rank Ensembles
Venue Conference on Empirical Methods in Natural Language Processing
Session Main Conference
Year 2014

We present power low rank ensembles (PLRE), a flexible framework for n-gram language modeling where ensembles of low rank matrices and tensors are used to obtain smoothed probability estimates of words in context. Our method can be understood as a generalization of n- gram modeling to non-integer n, and in- cludes standard techniques such as abso- lute discounting and Kneser-Ney smooth- ing as special cases. PLRE training is effi- cient and our approach outperforms state- of-the-art modified Kneser Ney baselines in terms of perplexity on large corpora as well as on BLEU score in a downstream machine translation task.