Paper: Scoring Coreference Partitions of Predicted Mentions: A Reference Implementation

ACL ID P14-2006
Title Scoring Coreference Partitions of Predicted Mentions: A Reference Implementation
Venue Annual Meeting of the Association of Computational Linguistics
Session Main Conference
Year 2014
Authors

The definitions of two coreference scoring metrics?B 3 and CEAF?are underspeci- fied with respect to predicted, as opposed to key (or gold) mentions. Several varia- tions have been proposed that manipulate either, or both, the key and predicted men- tions in order to get a one-to-one mapping. On the other hand, the metric BLANC was, until recently, limited to scoring partitions of key mentions. In this paper, we (i) ar- gue that mention manipulation for scoring predicted mentions is unnecessary, and po- tentially harmful as it could produce unin- tuitive results; (ii) illustrate the application of all these measures to scoring predicted mentions; (iii) make available an open- source, thoroughly-tested reference imple- mentation of the main coreference eval- uation measures; and (iv) rescor...