Paper: The GREC Challenges 2010: Overview and Evaluation Results

ACL ID W10-4226
Title The GREC Challenges 2010: Overview and Evaluation Results
Venue International Conference on Natural Language Generation
Session Main Conference
Year 2010

There were three GREC Tasks at Generation Challenges 2010: GREC-NER required participating systems to identify all people references in texts; for GRECNEG,systemsselectedcoreferencechains NEG,systemsselectedcoreferencechains for all people entities in texts; and GRECFull combined the NER and NEG tasks, i.e. systems identified and, if appropriate, replaced references to people in texts. Five teams submitted 10 systems in total, and we additionally created baseline systems for each task. Systems were evaluated automatically using a range of intrinsic metrics. In addition, systems were assessed by human judges using preference strength judgements. This report presents the evaluation results, along with descriptions of the three GREC tasks, the evaluation methods, ods, and the participating sy...