Paper: Inter-annotator Agreement for ERE annotation

ACL ID W14-2904
Title Inter-annotator Agreement for ERE annotation
Venue Workshop on Events: Definition, Detection, Coreference, and Representation
Year 2014

This paper describes a system for inter- annotator agreement analysis of ERE an- notation, focusing on entity mentions and how the higher-order annotations such as EVENTS are dependent on those entity mentions. The goal of this approach is to provide both (1) quantitative scores for the various levels of annotation, and (2) infor- mation about the types of annotation in- consistencies that might exist. While pri- marily designed for inter-annotator agree- ment, it can also be considered a system for evaluation of ERE annotation.