Paper: An Empirical Study Of Information Synthesis Task

ACL ID P04-1027
Title An Empirical Study Of Information Synthesis Task
Venue Annual Meeting of the Association of Computational Linguistics
Session Main Conference
Year 2004

This paper describes an empirical study of the “In- formation Synthesis” task, defined as the process of (given a complex information need) extracting, or- ganizing and inter-relating the pieces of information contained in a set of relevant documents, in order to obtain a comprehensive, non redundant report that satisfies the information need. Two main results are presented: a) the creation of an Information Synthesis testbed with 72 reports manually generated by nine subjects for eight com- plex topics with 100 relevant documents each; and b) an empirical comparison of similarity metrics be- tween reports, under the hypothesis that the best metric is the one that best distinguishes between manual and automatically generated reports. A met- ric based on key concepts overlap gives bette...