Paper: Word Embeddings through Hellinger PCA

ACL ID E14-1051
Title Word Embeddings through Hellinger PCA
Venue Annual Meeting of The European Chapter of The Association of Computational Linguistics
Session Main Conference
Year 2014

Word embeddings resulting from neural language models have been shown to be a great asset for a large variety of NLP tasks. However, such architecture might be difficult and time-consuming to train. Instead, we propose to drastically sim- plify the word embeddings computation through a Hellinger PCA of the word co- occurence matrix. We compare those new word embeddings with some well-known embeddings on named entity recognition and movie review tasks and show that we can reach similar or even better perfor- mance. Although deep learning is not re- ally necessary for generating good word embeddings, we show that it can provide an easy way to adapt embeddings to spe- cific tasks.