Paper: Is Question Answering Better than Information Retrieval? Towards a Task-Based Evaluation Framework for Question Series

ACL ID N07-1027
Title Is Question Answering Better than Information Retrieval? Towards a Task-Based Evaluation Framework for Question Series
Venue Human Language Technologies
Session Main Conference
Year 2007
Authors
  • Jimmy Lin (University of Maryland, College Park MD)

This paper introduces a novel evaluation framework for question series and em- ploys it to explore the effectiveness of QA and IR systems at addressing users’ infor- mation needs. The framework is based on the notion of recall curves, which char- acterize the amount of relevant informa- tion contained within a fixed-length text segment. Although it is widely assumed that QA technology provides more effi- cient access to information than IR sys- tems, our experiments show that a simple IR baseline is quite competitive. These re- sults help us better understand the role of NLP technology in QA systems and sug- gest directions for future research.