Paper: Native Judgments of Non-Native Usage: Experiments in Preposition Error Detection

ACL ID W08-1205
Title Native Judgments of Non-Native Usage: Experiments in Preposition Error Detection
Venue Coling 2008: Proceedings of the 2nd workshop on Information Retrieval for Question Answering
Session
Year 2008
Authors

Evaluation and annotation are two of the greatest challenges in developing NLP in- structional or diagnostic tools to mark grammar and usage errors in the writing of non-native speakers. Past approaches have commonly used only one rater to annotate a corpus of learner errors to compare to system output. In this paper, we show how using only one rater can skew system eval- uation and then we present a sampling ap- proach that makes it possible to evaluate a system more efficiently.