Paper: Nonparametric Method for Data-driven Image Captioning

ACL ID P14-2097
Title Nonparametric Method for Data-driven Image Captioning
Venue Annual Meeting of the Association of Computational Linguistics
Session Main Conference
Year 2014
Authors

We present a nonparametric density esti- mation technique for image caption gener- ation. Data-driven matching methods have shown to be effective for a variety of com- plex problems in Computer Vision. These methods reduce an inference problem for an unknown image to finding an exist- ing labeled image which is semantically similar. However, related approaches for image caption generation (Ordonez et al., 2011; Kuznetsova et al., 2012) are ham- pered by noisy estimations of visual con- tent and poor alignment between images and human-written captions. Our work addresses this challenge by estimating a word frequency representation of the vi- sual content of a query image. This al- lows us to cast caption generation as an extractive summarization problem. Our model strongly outperforms two s...