Paper: Connective-Based Measuring of the Inter-Annotator Agreement in the Annotation of Discourse in PDT

ACL ID C10-2089
Title Connective-Based Measuring of the Inter-Annotator Agreement in the Annotation of Discourse in PDT
Venue International Conference on Computational Linguistics
Session Poster Session
Year 2010
Authors

We present several ways of measuring the inter-annotator agreement in the on- going annotation of semantic inter-sen- tential discourse relations in the Prague Dependency Treebank (PDT). Two ways have been employed to overcome limitations of measuring the agreement on the exact location of the start/end points of the relations. Both methods – skipping one tree level in the start/end nodes, and the connective-based mea- sure – are focused on a recognition of the existence and of the type of the rela- tions, rather than on fixing the exact po- sitions of the start/end points of the con- necting arrows.