Paper: Experiments with crowdsourced re-annotation of a POS tagging data set

ACL ID P14-2062
Title Experiments with crowdsourced re-annotation of a POS tagging data set
Venue Annual Meeting of the Association of Computational Linguistics
Session Main Conference
Year 2014
Authors

Crowdsourcing lets us collect multiple an- notations for an item from several annota- tors. Typically, these are annotations for non-sequential classification tasks. While there has been some work on crowdsourc- ing named entity annotations, researchers have largely assumed that syntactic tasks such as part-of-speech (POS) tagging can- not be crowdsourced. This paper shows that workers can actually annotate sequen- tial data almost as well as experts. Fur- ther, we show that the models learned from crowdsourced annotations fare as well as the models learned from expert annota- tions in downstream tasks.