Paper: A Critical Reassessment of Evaluation Baselines for Speech Summarization

ACL ID P08-1054
Title A Critical Reassessment of Evaluation Baselines for Speech Summarization
Venue Annual Meeting of the Association of Computational Linguistics
Session Main Conference
Year 2008
Authors

We assess the current state of the art in speech summarization, by comparing a typical sum- marizer on two different domains: lecture data and the SWITCHBOARD corpus. Our re- sults cast significant doubt on the merits of this area’s accepted evaluation standards in terms of: baselines chosen, the correspondence of results to our intuition of what “summaries” should be, and the value of adding speech- related features to summarizers that already use transcripts from automatic speech recog- nition (ASR) systems. 1 Problem definition and related literature Speech is arguably the most basic, most natural form of human communication. The consistent demand for and increasing availability of spoken audio con- tent on web pages and other digital media should therefore come as no surprise. Al...