Paper: Intrinsic vs. Extrinsic Evaluation Measures for Referring Expression Generation

ACL ID P08-2050
Title Intrinsic vs. Extrinsic Evaluation Measures for Referring Expression Generation
Venue Annual Meeting of the Association of Computational Linguistics
Session Main Conference
Year 2008
Authors

In this paper we present research in which we apply (i) the kind of intrinsic evaluation met- rics that are characteristic of current compara- tive HLT evaluation, and (ii) extrinsic, human task-performance evaluations more in keeping with NLG traditions, to 15 systems implement- ing a language generation task. We analyse the evaluation results and find that there are no significant correlations between intrinsic and extrinsic evaluation measures for this task.