Paper: Distributional Semantics in Technicolor

ACL ID P12-1015
Title Distributional Semantics in Technicolor
Venue Annual Meeting of the Association of Computational Linguistics
Session Main Conference
Year 2012
Authors

Our research aims at building computational models of word meaning that are perceptually grounded. Using computer vision techniques, we build visual and multimodal distributional models and compare them to standard textual models. Our results show that, while visual models with state-of-the-art computer vision techniques perform worse than textual models in general tasks (accounting for semantic re- latedness), they are as good or better models of the meaning of words with visual correlates such as color terms, even in a nontrivial task that involves nonliteral uses of such words. Moreover, we show that visual and textual in- formation are tapping on different aspects of meaning, and indeed combining them in mul- timodal models often improves performance.