Paper: The Backtranslation Score: Automatic MT Evalution at the Sentence Level without Reference Translations

ACL ID P09-2034
Title The Backtranslation Score: Automatic MT Evalution at the Sentence Level without Reference Translations
Venue Annual Meeting of the Association of Computational Linguistics
Session Short Paper
Year 2009
Authors

Automatic tools for machine translation (MT) evaluation such as BLEU are well established, but have the drawbacks that they do not per- form well at the sentence level and that they presuppose manually translated reference texts. Assuming that the MT system to be evaluated can deal with both directions of a language pair, in this research we suggest to conduct automatic MT evaluation by determining the orthographic similarity between a back-trans- lation and the original source text. This way we eliminate the need for human translated reference texts. By correlating BLEU and back-translation scores with human judg- ments, it could be shown that the back- translation score gives an improved perfor- mance at the sentence level.