Paper: A New Dataset and Method for Automatically Grading ESOL Texts

ACL ID P11-1019
Title A New Dataset and Method for Automatically Grading ESOL Texts
Venue Annual Meeting of the Association of Computational Linguistics
Session Main Conference
Year 2011
Authors

We demonstrate how supervised discrimina- tive machine learning techniques can be used to automate the assessment of ‘English as a Second or Other Language’ (ESOL) examina- tion scripts. In particular, we use rank prefer- ence learning to explicitly model the grade re- lationships between scripts. A number of dif- ferent features are extracted and ablation tests are used to investigate their contribution to overall performance. A comparison between regression and rank preference models further supports our method. Experimental results on the first publically available dataset show that our system can achieve levels of performance close to the upper bound for the task, as de- fined by the agreement between human exam- iners on the same corpus. Finally, using a set of ‘outlier’ texts...