Paper: An Experiment On The Upper Bound Of Interjudge Agreement: The Case Of Tagging

ACL ID E99-1027
Title An Experiment On The Upper Bound Of Interjudge Agreement: The Case Of Tagging
Venue Annual Meeting of The European Chapter of The Association of Computational Linguistics
Session Main Conference
Year 1999
Authors

We investigate the controversial issue about the upper bound of interjudge agreement in the use of a low-level grammatical representation. Pessimistic views suggest that several percent of words in running text are undecidable in terms of part-of-speech categories. Our experiments with 55kW data give rea- son for optimism: linguists with only 30 hours' training apply the EngCG-2 mor- phological tags with almost 100% inter- judge agreement. 1 Orientation Linguistic analysers are developed for assign- ing linguistic descriptions to linguistic utterances. Linguistic descriptions are based on a fixed inven- tory of descriptors plus their usage principles: in short, a grammatical representation specified by linguists for the specific kind of analysis - e.g. morphological analysis, tagging, synt...