Paper: Exploiting Headword Dependency And Predictive Clustering For Language Modeling

ACL ID W02-1032
Title Exploiting Headword Dependency And Predictive Clustering For Language Modeling
Venue Conference on Empirical Methods in Natural Language Processing
Session Main Conference
Year 2002
Authors

This paper presents several practical ways of incorporating linguistic structure into language models. A headword detector is first applied to detect the headword of each phrase in a sentence. A permuted headword trigram model (PHTM) is then generated from the annotated corpus. Finally, PHTM is extended to a cluster PHTM (C-PHTM) by defining clusters for similar words in the corpus. We evaluated the proposed models on the realistic application of Japanese Kana-Kanji conversion. Experiments show that C-PHTM achieves 15% error rate reduction over the word trigram model. This demonstrates that the use of simple methods such as the headword trigram and predictive clustering can effectively capture long distance word dependency, and substantially outperform a word trigram model.