Paper: Probing The Lexicon In Evaluating Commercial MT Systems

ACL ID P97-1015
Title Probing The Lexicon In Evaluating Commercial MT Systems
Venue Annual Meeting of the Association of Computational Linguistics
Session Main Conference
Year 1997
Authors
  • Martin Volk (University of Zurich, Zurich Switzerland)

In the past the evaluation of machine trans- lation systems has focused on single sys- tem evaluations because there were only few systems available. But now there are several commercial systems for the same language pair. This requires new methods of comparative evaluation. In the paper we propose a black-box method for comparing the lexical coverage of MT systems. The method is based on lists of words from dif- ferent frequency classes. It is shown how these word lists can be compiled and used for testing. We also present the results of using our method on 6 MT systems that translate between English and German.