Paper: PORT: a Precision-Order-Recall MT Evaluation Metric for Tuning

ACL ID P12-1098
Title PORT: a Precision-Order-Recall MT Evaluation Metric for Tuning
Venue Annual Meeting of the Association of Computational Linguistics
Session Main Conference
Year 2012
Authors

Many machine translation (MT) evaluation metrics have been shown to correlate better with human judgment than BLEU. In principle, tuning on these metrics should yield better systems than tuning on BLEU. However, due to issues such as speed, requirements for linguistic resources, and optimization difficulty, they have not been widely adopted for tuning. This paper presents PORT 1 , a new MT evaluation metric which combines precision, recall and an ordering metric and which is primarily designed for tuning MT systems. PORT does not require external resources and is quick to compute. It has a better correlation with human judgment than BLEU. We compare PORT-tuned MT systems to BLEU-tuned baselines in five experimental conditions involving four language pairs. PORT tuning a...