Paper: A Salience Driven Approach To Robust Input Interpretation In Multimodal Conversational Systems

ACL ID H05-1028
Title A Salience Driven Approach To Robust Input Interpretation In Multimodal Conversational Systems
Venue Conference on Empirical Methods in Natural Language Processing
Session Main Conference
Year 2005
Authors

To improve the robustness in multimodal input interpretation, this paper presents a new salience driven approach. This approach is based on the observation that, during multimodal conversation, information from deictic gestures (e.g. , point or circle) on a graphical display can signal a part of the physical world (i.e. , representation of the domain and task) of the application which is salient during the communication. This salient part of the physical world will prime what users tend to communicate in speech and in turn can be used to constrain hypotheses for spoken language understanding, thus improving overall input interpretation. Our experimental results have indicated the potential of this approach in reducing word error rate and improving concept identification in multimodal conve...