Paper: Models of Translation Competitions

ACL ID P13-1139
Title Models of Translation Competitions
Venue Annual Meeting of the Association of Computational Linguistics
Session Main Conference
Year 2013

What do we want to learn from a trans- lation competition and how do we learn it with confidence? We argue that a dis- proportionate focus on ranking competi- tion participants has led to lots of differ- ent rankings, but little insight about which rankings we should trust. In response, we provide the first framework that allows an empirical comparison of different analy- ses of competition results. We then use this framework to compare several analyt- ical models on data from the Workshop on Machine Translation (WMT). 1 The WMT Translation Competition Every year, the Workshop on Machine Transla- tion (WMT) conducts a competition between ma- chine translation systems. The WMT organizers invite research groups to submit translation sys- tems in eight different tracks: Czech to/from En- glis...