Paper: Generalized Interpolation in Decision Tree LM

ACL ID P11-2109
Title Generalized Interpolation in Decision Tree LM
Venue Annual Meeting of the Association of Computational Linguistics
Session Main Conference
Year 2011

In the face of sparsity, statistical models are often interpolated with lower order (backoff) models, particularly in Language Modeling. In this paper, we argue that there is a rela- tion between the higher order and the backoff model that must be satisfied in order for the interpolation to be effective. We show that in n-gram models, the relation is trivially held, but in models that allow arbitrary clustering of context (such as decision tree models), this relation is generally not satisfied. Based on this insight, we also propose a generalization of linear interpolation which significantly im- proves the performance of a decision tree lan- guage model.