Paper: Improving Testsuites Via Instrumentation

ACL ID A00-1045
Title Improving Testsuites Via Instrumentation
Venue Annual Conference of the North American Chapter of the Association for Computational Linguistics
Session Main Conference
Year 2000

This paper explores the usefulness of a technique from software engineering, namely code instrumen- tation, for the development of large-scale natural language grammars. Information about the usage of grammar rules in test sentences is used to detect untested rules, redundant test sentences, and likely causes of overgeneration. Results show that less than half of a large-coverage grammar for German is actually tested by two large testsuites, and that i0- 30% of testing time is redundant. The methodology applied can be seen as a re-use of grammar writing knowledge for testsuite compilation.