Paper: Is Machine Translation Getting Better over Time?

ACL ID E14-1047
Title Is Machine Translation Getting Better over Time?
Venue Annual Meeting of The European Chapter of The Association of Computational Linguistics
Session Main Conference
Year 2014

Recent human evaluation of machine translation has focused on relative pref- erence judgments of translation quality, making it difficult to track longitudinal im- provements over time. We carry out a large-scale crowd-sourcing experiment to estimate the degree to which state-of-the- art performance in machine translation has increased over the past five years. To fa- cilitate longitudinal evaluation, we move away from relative preference judgments and instead ask human judges to provide direct estimates of the quality of individ- ual translations in isolation from alternate outputs. For seven European language pairs, our evaluation estimates an aver- age 10-point improvement to state-of-the- art machine translation between 2007 and 2012, with Czech-to-English translation standing out as t...