Paper: Assessing the Impact of Translation Errors on Machine Translation Quality with Mixed-effects Models

ACL ID D14-1172
Title Assessing the Impact of Translation Errors on Machine Translation Quality with Mixed-effects Models
Venue Conference on Empirical Methods in Natural Language Processing
Session Main Conference
Year 2014
Authors

Learning from errors is a crucial aspect of improving expertise. Based on this no- tion, we discuss a robust statistical frame- work for analysing the impact of different error types on machine translation (MT) output quality. Our approach is based on linear mixed-effects models, which allow the analysis of error-annotated MT out- put taking into account the variability in- herent to the specific experimental setting from which the empirical observations are drawn. Our experiments are carried out on different language pairs involving Chi- nese, Arabic and Russian as target lan- guages. Interesting findings are reported, concerning the impact of different error types both at the level of human perception of quality and with respect to performance results measured with automatic metrics.