Paper: A Unified Framework For Automatic Evaluation Using N-Gram Co-Occurrence Statistics

ACL ID P04-1078
Title A Unified Framework For Automatic Evaluation Using N-Gram Co-Occurrence Statistics
Venue Annual Meeting of the Association of Computational Linguistics
Session Main Conference
Year 2004
Authors

In this paper we propose a unified framework for automatic evaluation of NLP applications using N-gram co-occurrence statistics. The automatic evaluation metrics proposed to date for Machine Translation and Automatic Summarization are particular instances from the family of metrics we propose. We show that different members of the same family of metrics explain best the variations obtained with human evaluations, according to the application being evaluated (Machine Translation, Automatic Summarization, and Automatic Question Answering) and the evaluation guidelines used by humans for evaluating such applications.