Paper: Automated Vocabulary Acquisition and Interpretation in Multimodal Conversational Systems

ACL ID P07-1047
Title Automated Vocabulary Acquisition and Interpretation in Multimodal Conversational Systems
Venue Annual Meeting of the Association of Computational Linguistics
Session Main Conference
Year 2007
Authors

Motivated by psycholinguistic findings that eye gaze is tightly linked to human lan- guage production, we developed an unsuper- vised approach based on translation models to automatically learn the mappings between words and objects on a graphic display dur- ing human machine conversation. The ex- perimental results indicate that user eye gaze can provide useful information to establish such mappings, which have important impli- cations in automatically acquiring and inter- preting user vocabularies for conversational systems.