Paper: Dialogue Segmentation with Large Numbers of Volunteer Internet Annotators

ACL ID P09-1101
Title Dialogue Segmentation with Large Numbers of Volunteer Internet Annotators
Venue Annual Meeting of the Association of Computational Linguistics
Session Main Conference
Year 2009
Authors

This paper shows the results of an experiment in dialogue segmentation. In this experiment, segmentation was done on a level of analysis similar to adjacency pairs. The method of annotation was somewhat novel: volunteers were invited to participate over the Web, and their responses were aggregated using a simple voting method. Though volunteers received a minimum of training, the aggregated responses of the group showed very high agreement with expert opinion. The group, as a unit, performed at the top of the list of annotators, and in many cases performed as well as or better than the best annotator.