Paper: Attribute Selection for Referring Expression Generation: New Algorithms and Evaluation Methods

ACL ID W08-1108
Title Attribute Selection for Referring Expression Generation: New Algorithms and Evaluation Methods
Venue International Conference on Natural Language Generation
Session Main Conference
Year 2008
Authors

Referring expression generation has recently been the subject of the first Shared Task Chal- lenge in NLG. In this paper, we analyse the systems that participated in the Challenge in terms of their algorithmic properties, compar- ing new techniques to classic ones, based on results from a new human task-performance experiment and from the intrinsic measures that were used in the Challenge. We also con- sider the relationship between different eval- uation methods, showing that extrinsic task- performance experiments and intrinsic evalu- ation methods yield results that are not signif- icantly correlated. We argue that this high- lights the importance of including extrinsic evaluation methods in comparative NLG eval- uations.