Paper: Utility Evaluation of Cross-document Information Extraction

ACL ID N10-1036
Title Utility Evaluation of Cross-document Information Extraction
Venue Human Language Technologies
Session Main Conference
Year 2010

We describe a utility evaluation to determine whether cross-document information extrac- tion (IE) techniques measurably improve user performance in news summary writing. Two groups of subjects were asked to perform the same time-restricted summary writing tasks, reading news under different conditions: with no IE results at all, with traditional single- document IE results, and with cross-document IE results. Our results show that, in compari- son to using source documents only, the qual- ity of summary reports assembled using IE results, especially from cross-document IE, was significantly better and user satisfaction was higher. We also compare the impact of different user groups on the results.