Paper: Kernel-Based Approach For Automatic Evaluation Of Natural Language Generation Technologies: Application To Automatic Summarization

ACL ID H05-1019
Title Kernel-Based Approach For Automatic Evaluation Of Natural Language Generation Technologies: Application To Automatic Summarization
Venue Conference on Empirical Methods in Natural Language Processing
Session Main Conference
Year 2005
Authors

In order to promote the study of auto- matic summarization and translation, we need an accurate automatic evaluation method that is close to human evalua- tion. In this paper, we present an eval- uation method that is based on convolu- tion kernels that measure the similarities between texts considering their substruc- tures. We conducted an experiment us- ing automatic summarization evaluation data developed for Text Summarization Challenge 3 (TSC-3). A comparison with conventional techniques shows that our method correlates more closely with hu- man evaluations and is more robust.