Paper: RED: A Reference Dependency Based MT Evaluation Metric

ACL ID C14-1193
Title RED: A Reference Dependency Based MT Evaluation Metric
Venue International Conference on Computational Linguistics
Session Main Conference
Year 2014

Most of the widely-used automatic evaluation metrics consider only the local fragments of the references and translations, and they ignore the evaluation on the syntax level. Current syntax- based evaluation metrics try to introduce syntax information but suffer from the poor pars- ing results of the noisy machine translations. To alleviate this problem, we propose a novel dependency-based evaluation metric which only employs the dependency information of the ref- erences. We use two kinds of reference dependency structures: headword chain to capture the long distance dependency information, and fixed and floating structures to capture the local con- tinuous ngram. Experiment results show that our metric achieves higher correlations with human judgments than BLEU, TER and HWCM on WMT 2012 ...