Paper: They Can Help: Using Crowdsourcing to Improve the Evaluation of Grammatical Error Detection Systems

ACL ID P11-2089
Title They Can Help: Using Crowdsourcing to Improve the Evaluation of Grammatical Error Detection Systems
Venue Annual Meeting of the Association of Computational Linguistics
Session Main Conference
Year 2011
Authors

Despite the rising interest in developing gram- matical error detection systems for non-native speakers of English, progress in the field has been hampered by a lack of informative met- rics and an inability to directly compare the performance of systems developed by differ- ent researchers. In this paper we address these problems by presenting two evaluation methodologies, both based on a novel use of crowdsourcing. 1 Motivation and Contributions One of the fastest growing areas in need of NLP tools is the field of grammatical error detection for learners of English as a Second Language (ESL). According to Guo and Beckett (2007), “over a bil- lion people speak English as their second or for- eign language.” This high demand has resulted in many NLP research papers on the topic, a Synt...