Paper: GENEVAL: A Proposal For Shared-Task Evaluation In NLG

ACL ID W06-1422
Title GENEVAL: A Proposal For Shared-Task Evaluation In NLG
Venue International Conference on Natural Language Generation
Session Main Conference
Year 2006

We propose to organise a series of shared- task NLG events, where participants are asked to build systems with similar in- put/output functionalities, and these sys- tems are evaluated with a range of differ- ent evaluation techniques. The main pur- pose of these events is to allow us to com- pare different evaluation techniques, by correlating the results of different evalua- tions on the systems entered in the events. 1 Background Evaluation is becoming increasingly important in Natural Language Generation (NLG), as in most otherareasofNaturalLanguageProcessing(NLP). NLG systems can be evaluated in many differ- ent ways, with different associated resource re- quirements. For example, a large-scale task- effectiveness study with human subjects could last over a year and cost more than US$...