Paper: ReferItGame: Referring to Objects in Photographs of Natural Scenes

ACL ID D14-1086
Title ReferItGame: Referring to Objects in Photographs of Natural Scenes
Venue Conference on Empirical Methods in Natural Language Processing
Session Main Conference
Year 2014
Authors

In this paper we introduce a new game to crowd-source natural language referring expressions. By designing a two player game, we can both collect and verify refer- ring expressions directly within the game. To date, the game has produced a dataset containing 130,525 expressions, referring to 96,654 distinct objects, in 19,894 pho- tographs of natural scenes. This dataset is larger and more varied than previous REG datasets and allows us to study referring expressions in real-world scenes. We pro- vide an in depth analysis of the resulting dataset. Based on our findings, we design a new optimization based model for gen- erating referring expressions and perform experimental evaluations on 3 test sets.