Paper: Shared Logistic Normal Distributions for Soft Parameter Tying in Unsupervised Grammar Induction

ACL ID N09-1009
Title Shared Logistic Normal Distributions for Soft Parameter Tying in Unsupervised Grammar Induction
Venue Human Language Technologies
Session Main Conference
Year 2009
Authors

We present a family of priors over probabilis- ticgrammarweights, calledthesharedlogistic normal distribution. This family extends the partitioned logistic normal distribution, en- abling factored covariance between the prob- abilities of different derivation events in the probabilistic grammar, providing a new way to encode prior knowledge about an unknown grammar. We describe a variational EM al- gorithm for learning a probabilistic grammar based on this family of priors. We then experi- ment with unsupervised dependency grammar induction and show significant improvements using our model for both monolingual learn- ing and bilingual learning with a non-parallel, multilingual corpus.