Paper: Training Neural Network Language Models On Very Large Corpora

ACL ID H05-1026
Title Training Neural Network Language Models On Very Large Corpora
Venue Conference on Empirical Methods in Natural Language Processing
Session Main Conference
Year 2005
Authors

During the last years there has been grow- ing interest in using neural networks for language modeling. In contrast to the well known back-off n-gram language models, the neural network approach attempts to overcome the data sparseness problem by performing the estimation in a continuous space. This type of language model was mostly used for tasks for which only a very limited amount of in-domain training data is available. In this paper we present new algorithms to train a neural network language model on very large text corpora. This makes pos- sible the use of the approach in domains where several hundreds of millions words of texts are available. The neural network language model is evaluated in a state-of- the-art real-time continuous speech recog- nizer for French Broadcast News. Wor...