Paper: Evaluating the Inferential Utility of Lexical-Semantic Resources

ACL ID E09-1064
Title Evaluating the Inferential Utility of Lexical-Semantic Resources
Venue Annual Meeting of The European Chapter of The Association of Computational Linguistics
Session Main Conference
Year 2009

Lexical-semantic resources are used ex- tensively for applied semantic inference, yet a clear quantitative picture of their current utility and limitations is largely missing. We propose system- and application-independent evaluation and analysis methodologies for resources’ per- formance, and systematically apply them to seven prominent resources. Our find- ings identify the currently limited recall of available resources, and indicate the po- tential to improve performance by exam- ining non-standard relation types and by distilling the output of distributional meth- ods. Further, our results stress the need to include auxiliary information regarding the lexical and logical contexts in which a lexical inference is valid, as well as its prior validity likelihood.