Paper: Active Learning with Amazon Mechanical Turk

ACL ID D11-1143
Title Active Learning with Amazon Mechanical Turk
Venue Conference on Empirical Methods in Natural Language Processing
Session Main Conference
Year 2011

Supervised classification needs large amounts of annotated training data that is expensive to create. Two approaches that reduce the cost of annotation are active learning and crowd- sourcing. However, these two approaches have not been combined successfully to date. We evaluate the utility of active learning in crowdsourcing on two tasks, named entity recognition and sentiment detection, and show that active learning outperforms random selec- tion of annotation examples in a noisy crowd- sourcing scenario.