Paper: Evaluation In The ARPA Machine Translation Program: 1993 Methodology

ACL ID H94-1024
Title Evaluation In The ARPA Machine Translation Program: 1993 Methodology
Venue Human Language Technologies
Session Main Conference
Year 1994
Authors

In the second year of evaluations of the ARPA HLT Machine Translation (MT) Initiative, methodologies developed and tested in 1992 were applied to the 1993 MT test runs. The current methodology optimizes the inherently subjective judgments on translation accuracy and quality by channeling the judgments of non-translators into many data points which reflect both the comparison of the performance of the research MT systems with production MT systems and against the performance of novice translators. This paper discusses the three evaluation methods used in the 1993 evaluation, the results of the evaluations, and preliminary characterizations of the Winter 1994 evaluation, now underway. The efforts under discussion focus on measuring the.progress of core MT technology and increasing the sensit...