Paper: An Adaptive Approach To Collecting Multimodal Input

ACL ID P03-2005
Title An Adaptive Approach To Collecting Multimodal Input
Venue Annual Meeting of the Association of Computational Linguistics
Session Main Conference
Year 2003
  • Anurag Gupta (University of New South Wales, Sydney Australia)

Multimodal dialogue systems allow users to input information in multiple modali- ties. These systems can handle simultane- ous or sequential composite multimodal input. Different coordination schemes re- quire such systems to capture, collect and integrate user input in different modali- ties, and then respond to a joint interpreta- tion. We performed a study to understand the variability of input in multimodal dia- logue systems and to evaluate methods to perform the collection of input informa- tion. An enhancement in the form of in- corporation of a dynamic time window to a multimodal input fusion module was proposed in the study. We found that the enhanced module provides superior tem- poral characteristics and robustness when compared to previous methods.