Paper: Recurrent Neural Networks for Word Alignment Model

ACL ID P14-1138
Title Recurrent Neural Networks for Word Alignment Model
Venue Annual Meeting of the Association of Computational Linguistics
Session Main Conference
Year 2014

This study proposes a word alignment model based on a recurrent neural net- work (RNN), in which an unlimited alignment history is represented by re- currently connected hidden layers. We perform unsupervised learning using noise-contrastive estimation (Gutmann and Hyv?arinen, 2010; Mnih and Teh, 2012), which utilizes artificially generated negative samples. Our alignment model is directional, similar to the generative IBM models (Brown et al., 1993). To overcome this limitation, we encourage agreement between the two directional models by introducing a penalty function that en- sures word embedding consistency across two directional models during training. The RNN-based model outperforms the feed-forward neural network-based model (Yang et al., 2013) as well as the IBM Model 4 under Japan...