Paper: Automatic Evaluation Of Summaries Using N-Gram Co-Occurrence Statistics

ACL ID N03-1020
Title Automatic Evaluation Of Summaries Using N-Gram Co-Occurrence Statistics
Venue Human Language Technologies
Session Main Conference
Year 2003
Authors

Following the recent adoption by the machine translation community of automatic evalua- tion using the BLEU/NIST scoring process, we conduct an in-depth study of a similar idea for evaluating summaries. The results show that automatic evaluation using unigram co- occurrences between summary pairs correlates surprising well with human evaluations, based on various statistical metrics; while direct ap- plication of the BLEU evaluation procedure does not always give good results.