Paper: Competitive Generative Models With Structure Learning For NLP Classification Tasks

ACL ID W06-1668
Title Competitive Generative Models With Structure Learning For NLP Classification Tasks
Venue Conference on Empirical Methods in Natural Language Processing
Session Main Conference
Year 2006
Authors

In this paper we show that generative models are competitive with and some- times superior to discriminative models, when both kinds of models are allowed to learn structures that are optimal for dis- crimination. In particular, we compare Bayesian Networks and Conditional log- linear models on two NLP tasks. We ob- serve that when the structure of the gen- erative model encodes very strong inde- pendence assumptions (a la Naive Bayes), a discriminative model is superior, but when the generative model is allowed to weaken these independence assumptions via learning a more complex structure, it can achieve very similar or better perfor- mance than a corresponding discrimina- tive model. In addition, as structure learn- ing for generative models is far more ef- ficient, they may be preferabl...