Paper: Contrastive Estimation: Training Log-Linear Models On Unlabeled Data

ACL ID P05-1044
Title Contrastive Estimation: Training Log-Linear Models On Unlabeled Data
Venue Annual Meeting of the Association of Computational Linguistics
Session Main Conference
Year 2005
Authors

Conditional random fields (Lafferty et al. , 2001) are quite effective at sequence labeling tasks like shal- low parsing (Sha and Pereira, 2003) and named- entity extraction (McCallum and Li, 2003). CRFs are log-linear, allowing the incorporation of arbi- trary features into the model. To train on unlabeled data, we require unsupervised estimation methods for log-linear models; few exist. We describe a novel approach, contrastive estimation. We show that the new technique can be intuitively understood as ex- ploiting implicit negative evidence and is computa- tionally efficient. Applied to a sequence labeling problem—POS tagging given a tagging dictionary and unlabeled text—contrastive estimation outper- forms EM (with the same feature set), is more robust to degradations of the dictio...