Paper: What's There To Talk About? A Multi-Modal Model Of Referring Behavior In The Presence Of Shared Visual Information

ACL ID E06-3001
Title What's There To Talk About? A Multi-Modal Model Of Referring Behavior In The Presence Of Shared Visual Information
Venue Annual Meeting of The European Chapter of The Association of Computational Linguistics
Session Main Conference
Year 2006
Authors

This paper describes the development of a rule-based computational model that describes how a feature-based representa- tion of shared visual information com- bines with linguistic cues to enable effec- tive reference resolution. This work ex- plores a language-only model, a visual- only model, and an integrated model of reference resolution and applies them to a corpus of transcribed task-oriented spo- ken dialogues. Preliminary results from a corpus-based analysis suggest that inte- grating information from a shared visual environment can improve the perform- ance and quality of existing discourse- based models of reference resolution.