Paper: Conditional Modality Fusion for Coreference Resolution

ACL ID P07-1045
Title Conditional Modality Fusion for Coreference Resolution
Venue Annual Meeting of the Association of Computational Linguistics
Session Main Conference
Year 2007

Non-verbal modalities such as gesture can improve processing of spontaneous spoken language. For example, similar hand ges- tures tend to predict semantic similarity, so features that quantify gestural similarity can improve semantic tasks such as coreference resolution. However, not all hand move- ments are informative gestures; psycholog- ical research has shown that speakers are more likely to gesture meaningfully when their speech is ambiguous. Ideally, one would attend to gesture only in such cir- cumstances, and ignore other hand move- ments. We present conditional modality fusion, which formalizes this intuition by treating the informativeness of gesture as a hidden variable to be learned jointly with the class label. Applied to coreference resolution, conditional modality fusion si...