Paper: Information Structure Prediction for Visual-world Referring Expressions

ACL ID E14-1055
Title Information Structure Prediction for Visual-world Referring Expressions
Venue Annual Meeting of The European Chapter of The Association of Computational Linguistics
Session Main Conference
Year 2014
Authors

We investigate the order of mention for objects in relational descriptions in visual scenes. Existing work in the visual do- main focuses on content selection for text generation and relies primarily on tem- plates to generate surface realizations from underlying content choices. In contrast, we seek to clarify the influence of visual perception on the linguistic form (as op- posed to the content) of descriptions, mod- eling the variation in and constraints on the surface orderings in a description. We find previously-unknown effects of the vi- sual characteristics of objects; specifically, when a relational description involves a vi- sually salient object, that object is more likely to be mentioned first. We conduct a detailed analysis of these patterns using logistic regression, and also...