Paper: Language Modeling with Power Low Rank Ensembles

ACL ID D14-1158
Title Language Modeling with Power Low Rank Ensembles
Venue Conference on Empirical Methods in Natural Language Processing
Session Main Conference
Year 2014
Authors

We present power low rank ensembles (PLRE), a flexible framework for n-gram language modeling where ensembles of low rank matrices and tensors are used to obtain smoothed probability estimates of words in context. Our method can be understood as a generalization of n- gram modeling to non-integer n, and in- cludes standard techniques such as abso- lute discounting and Kneser-Ney smooth- ing as special cases. PLRE training is effi- cient and our approach outperforms state- of-the-art modified Kneser Ney baselines in terms of perplexity on large corpora as well as on BLEU score in a downstream machine translation task.