Paper: Extreme Extraction – Machine Reading in a Week

ACL ID D11-1133
Title Extreme Extraction – Machine Reading in a Week
Venue Conference on Empirical Methods in Natural Language Processing
Session Main Conference
Year 2011

We report on empirical results in extreme extraction. It is extreme in that (1) from re- ceipt of the ontology specifying the target concepts and relations, development is li- mited to one week and that (2) relatively little training data is assumed. We are able to surpass human recall and achieve an F1 of 0.51 on a question-answering task with less than 50 hours of effort using a hybrid approach that mixes active learning, boot- strapping, and limited (5 hours) manual rule writing. We compare the performance of three systems: extraction with handwrit- ten rules, bootstrapped extraction, and a combination. We show that while the recall of the handwritten rules surpasses that of the learned system, the learned system is able to improve the overall recall and F1.