Paper: Evaluating Distributional Models of Semantics for Syntactically Invariant Inference

ACL ID E12-1005
Title Evaluating Distributional Models of Semantics for Syntactically Invariant Inference
Venue Annual Meeting of The European Chapter of The Association of Computational Linguistics
Session Main Conference
Year 2012
Authors

A major focus of current work in distri- butional models of semantics is to con- struct phrase representations composition- ally from word representations. However, the syntactic contexts which are modelled are usually severely limited, a fact which is reflected in the lexical-level WSD-like evaluation methods used. In this paper, we broaden the scope of these models to build sentence-level representations, and argue that phrase representations are best eval- uated in terms of the inference decisions that they support, invariant to the partic- ular syntactic constructions used to guide composition. We propose two evaluation methods in relation classification and QA which reflect these goals, and apply several recent compositional distributional models to the tasks. We find that the models ...