Paper: An Exploration of Eye Gaze in Spoken Language Processing for Multimodal Conversational Interfaces

ACL ID N07-1036
Title An Exploration of Eye Gaze in Spoken Language Processing for Multimodal Conversational Interfaces
Venue Human Language Technologies
Session Main Conference
Year 2007
Authors

Motivated by psycholinguistic findings, we are currently investigating the role of eye gaze in spoken language understand- ing for multimodal conversational sys- tems. Our assumption is that, during hu- man machine conversation, a user’s eye gaze on the graphical display indicates salient entities on which the user’s atten- tion is focused. The specific domain infor- mation about the salient entities is likely to be the content of communication and therefore can be used to constrain speech hypotheses and help language understand- ing. Based on this assumption, this paper describes an exploratory study that incor- porates eye gaze in salience modeling for spoken language processing. Our empiri- cal results show that eye gaze has a poten- tial in improving automated language pro- cessing...