Paper: HyTER: Meaning-Equivalent Semantics for Translation Evaluation

ACL ID N12-1017
Title HyTER: Meaning-Equivalent Semantics for Translation Evaluation
Venue Annual Conference of the North American Chapter of the Association for Computational Linguistics
Session Main Conference
Year 2012

It is common knowledge that translation is an ambiguous, 1-to-n mapping process, but to date, our community has produced no em- pirical estimates of this ambiguity. We have developed an annotation tool that enables us to create representations that compactly en- code an exponential number of correct trans- lations for a sentence. Our findings show that naturally occurring sentences have billions of translations. Having access to such large sets of meaning-equivalent translations enables us to develop a new metric, HyTER, for transla- tion accuracy. We show that our metric pro- vides better estimates of machine and human translation accuracy than alternative evalua- tion metrics. 1 Motivation During the last decade, automatic evaluation met- rics (Papineni et al., 2002; Snover et al., 2006;...