Paper: Semantics-Based Representation For Multimodal Interpretation In Conversational Systems

ACL ID C02-1035
Title Semantics-Based Representation For Multimodal Interpretation In Conversational Systems
Venue International Conference on Computational Linguistics
Session Main Conference
Year 2002
Authors
  • Joyce Chai (IBM T.J. Watson Research Center, Yorktown Heights NY)

To support context-based multimodal interpre- tation in conversational systems, we have devel- oped a semantics-based representation to capture salient information from user inputs and the overall conversation. In particular, we present three unique characteristics: fine- grained semantic models, flexible composition of feature structures, and consistent representa- tion at multiple levels. This representation allows our system to use rich contexts to resolve ambiguities, infer unspecified information, and improve multimodal alignment. As a result, our system is able to enhance understanding of mul- timodal inputs including those abbreviated, imprecise, or complex ones.