Paper: Using A Corpus Of Sentence Orderings Defined By Many Experts To Evaluate Metrics Of Coherence For Text Structuring

ACL ID W05-1621
Title Using A Corpus Of Sentence Orderings Defined By Many Experts To Evaluate Metrics Of Coherence For Text Structuring
Venue ENLG
Session Main Conference
Year 2005
Authors

This paper addresses two previously unresolved is- sues in the automatic evaluation of Text Structuring (TS) in Natural Language Generation (NLG). First, we describe how to verify the generality of an exist- ing collection of sentence orderings defined by one domain expert using data provided by additional experts. Second, a general evaluation methodol- ogy is outlined which investigates the previously unaddressed possibility that there may exist many optimal solutions for TS in the employed domain.