Paper: A Scalable Probabilistic Classifier for Language Modeling

ACL ID P11-2110
Title A Scalable Probabilistic Classifier for Language Modeling
Venue Annual Meeting of the Association of Computational Linguistics
Session Main Conference
Year 2011

We present a novel probabilistic classifier, which scales well to problems that involve a large number of classes and require training on large datasets. A prominent example of such a problem is language modeling. Our classifier is based on the assumption that each feature is associated with a predictive strength, which quantifies how well the feature can predict the class by itself. The predictions of individual features can then be combined according to their predictive strength, resulting in a model, whose parameters can be reliably and effi- ciently estimated. We show that a generative language model based on our classifier consis- tently matches modified Kneser-Ney smooth- ing and can outperform it if sufficiently rich features are incorporated.