Paper: Bayesian Grammar Induction For Language Modeling

ACL ID P95-1031
Title Bayesian Grammar Induction For Language Modeling
Venue Annual Meeting of the Association of Computational Linguistics
Session Main Conference
Year 1995

We describe a corpus-based induction algo- rithm for probabilistic context-free gram- mars. The algorithm employs a greedy heuristic search within a Bayesian frame- work, and a post-pass using the Inside- Outside algorithm. We compare the per- formance of our algorithm to n-gram mo- dels and the Inside-Outside algorithm in three language modeling tasks. In two of the tasks, the training data is generated by a probabilistic context-free grammar and in both tasks our algorithm outperforms the other techniques. The third task involves naturally-occurring data, and in this task our algorithm does not perform as well as n-gram models but vastly outperforms the Inside-Outside algorithm.