Paper: Kneser-Ney Smoothing on Expected Counts

ACL ID P14-1072
Title Kneser-Ney Smoothing on Expected Counts
Venue Annual Meeting of the Association of Computational Linguistics
Session Main Conference
Year 2014

Widely used in speech and language pro- cessing, Kneser-Ney (KN) smoothing has consistently been shown to be one of the best-performing smoothing methods. However, KN smoothing assumes integer counts, limiting its potential uses?for ex- ample, inside Expectation-Maximization. In this paper, we propose a generaliza- tion of KN smoothing that operates on fractional counts, or, more precisely, on distributions over counts. We rederive all the steps of KN smoothing to operate on count distributions instead of integral counts, and apply it to two tasks where KN smoothing was not applicable before: one in language model adaptation, and the other in word alignment. In both cases, our method improves performance signifi- cantly.