Paper: Instance-based Evaluation of Entailment Rule Acquisition

ACL ID P07-1058
Title Instance-based Evaluation of Entailment Rule Acquisition
Venue Annual Meeting of the Association of Computational Linguistics
Session Main Conference
Year 2007

Obtaining large volumes of inference knowl- edge, such as entailment rules, has become a major factor in achieving robust seman- tic processing. While there has been sub- stantial research on learning algorithms for such knowledge, their evaluation method- ology has been problematic, hindering fur- ther research. We propose a novel evalua- tion methodology for entailment rules which explicitly addresses their semantic proper- ties and yields satisfactory human agreement levels. The methodology is used to compare two state of the art learning algorithms, ex- posing critical issues for future progress.