Paper: Language Models Based on Semantic Composition

ACL ID D09-1045
Title Language Models Based on Semantic Composition
Venue Conference on Empirical Methods in Natural Language Processing
Session Main Conference
Year 2009

In this paper we propose a novel statistical language model to capture long-range se- mantic dependencies. Specifically, we ap- plytheconceptofsemanticcompositionto the problem of constructing predictive his- tory representations for upcoming words. We also examine the influence of the un- derlying semantic space on the composi- tion task by comparing spatial semantic representations against topic-based ones. The composition models yield reductions in perplexity when combined with a stan- dard n-gram language model over the n-gram model alone. We also obtain per- plexity reductions when integrating our models with a structured language model.