Paper: Unsupervised Model Adaptation using Information-Theoretic Criterion

ACL ID N10-1022
Title Unsupervised Model Adaptation using Information-Theoretic Criterion
Venue Human Language Technologies
Session Main Conference
Year 2010
Authors

In this paper we propose a novel general framework for unsupervised model adapta- tion. Our method is based on entropy which has been used previously as a regularizer in semi-supervised learning. This technique in- cludes another term which measures the sta- bility of posteriors w.r.t model parameters, in addition to conditional entropy. The idea is to use parameters which result in both low con- ditional entropy and also stable decision rules. As an application, we demonstrate how this framework can be used for adjusting language model interpolation weight for speech recog- nition task to adapt from Broadcast news data to MIT lecture data. We show how the new technique can obtain comparable performance to completely supervised estimation of inter- polation parameters.