Paper: Situated Incremental Natural Language Understanding using a Multimodal, Linguistically-driven Update Model

ACL ID C14-1170
Title Situated Incremental Natural Language Understanding using a Multimodal, Linguistically-driven Update Model
Venue International Conference on Computational Linguistics
Session Main Conference
Year 2014
Authors

A common site of language use is interactive dialogue between two people situated together in shared time and space. In this paper, we present a statistical model for understanding natural human language that works incrementally (i.e., does not wait until the end of an utterance to begin processing), and is grounded by linking semantic entities with objects in a shared space. We describe our model, show how a semantic meaning representation is grounded with properties of real-world objects, and further show that it can ground with embodied, interactive cues such as pointing gestures or eye gaze.