Paper: Improving machine translation by training against an automatic semantic frame based evaluation metric

ACL ID P13-2067
Title Improving machine translation by training against an automatic semantic frame based evaluation metric
Venue Annual Meeting of the Association of Computational Linguistics
Session Short Paper
Year 2013
Authors

We present the first ever results show- ing that tuning a machine translation sys- tem against a semantic frame based ob- jective function, MEANT, produces more robustly adequate translations than tun- ing against BLEU or TER as measured across commonly used metrics and human subjective evaluation. Moreover, for in- formal web forum data, human evalua- tors preferredMEANT-tuned systems over BLEU- or TER-tuned systems by a sig- nificantly wider margin than that for for- mal newswire?even though automatic se- mantic parsing might be expected to fare worse on informal language. We argue that by preserving themeaning of the trans- lations as captured by semantic frames right in the training process, an MT sys- tem is constrained to make more accu- rate choices of both lexical and reorder- ing ...