Paper: Efficient Computation of Entropy Gradient for Semi-Supervised Conditional Random Fields

ACL ID N07-2028
Title Efficient Computation of Entropy Gradient for Semi-Supervised Conditional Random Fields
Venue Human Language Technologies
Session Short Paper
Year 2007
Authors

Entropyregularizationisastraightforward and successful method of semi-supervised learning that augments the traditional con- ditional likelihood objective function with an additional term that aims to minimize the predicted label entropy on unlabeled data. It has previously been demonstrated to provide positive results in linear-chain CRFs, but the published method for cal- culating the entropy gradient requires sig- nificantly more computation than super- vised CRF training. This paper presents a new derivation and dynamic program for calculating the entropy gradient that is significantly more efficient—having the same asymptotic time complexity as su- pervised CRF training. We also present efficient generalizations of this method for calculating the label entropy of all sub-sequences, ...