Paper: Improving Language Models By Clustering Training Sentences

ACL ID A94-1010
Title Improving Language Models By Clustering Training Sentences
Venue Applied Natural Language Processing Conference
Session Main Conference
Year 1994

Many of the kinds of language model used in speech understanding suffer from imper- fect modeling of intra-sentential contextual influences. I argue that this problem can be addressed by clustering the sentences in a training corpus automatically into subcor- pora on the criterion of entropy reduction, and calculating separate language model parameters for each cluster. This kind of clustering offers a way to represent impor- tant contextual effects and can therefore significantly improve the performance of a model. It also offers a reasonably auto- matic means to gather evidence on whether a more complex, context-sensitive model using the same general kind of linguistic in- formation is likely to reward the effort that would be required to develop it: if cluster- ing improves the performa...