Paper: Better Word Representations with Recursive Neural Networks for Morphology

ACL ID W13-3512
Title Better Word Representations with Recursive Neural Networks for Morphology
Venue International Conference on Computational Natural Language Learning
Session Main Conference
Year 2013
Authors

Vector-space word representations have been very successful in recent years at im- proving performance across a variety of NLP tasks. However, common to most existing work, words are regarded as in- dependent entities without any explicit re- lationship among morphologically related words being modeled. As a result, rare and complex words are often poorly estimated, and all unknown words are represented in a rather crude way using only one or a few vectors. This paper addresses this shortcoming by proposing a novel model that is capable of building representations for morphologically complex words from their morphemes. We combine recursive neural networks (RNNs), where each mor- pheme is a basic unit, with neural language models (NLMs) to consider contextual information in learning morphol...