Paper: Goodness: A Method for Measuring Machine Translation Confidence

ACL ID P11-1022
Title Goodness: A Method for Measuring Machine Translation Confidence
Venue Annual Meeting of the Association of Computational Linguistics
Session Main Conference
Year 2011
Authors

State-of-the-art statistical machine translation (MT) systems have made significant progress towards producing user-acceptable translation output. However, there is still no efficient way for MT systems to inform users which words are likely translated correctly and how confident it is about the whole sentence. We propose a novel framework to predict word- level and sentence-level MT errors with a large number of novel features. Experimental re- sults show that the MT error prediction accu- racy is increased from 69.1 to 72.2 in F-score. The Pearson correlation between the proposed confidence measure and the human-targeted translation edit rate (HTER) is 0.6. Improve- ments between 0.4 and 0.9 TER reduction are obtained with the n-best list reranking task us- ing the proposed confidence me...