Paper: Choosing an Evaluation Metric for Parser Design

ACL ID N12-2006
Title Choosing an Evaluation Metric for Parser Design
Venue Annual Conference of the North American Chapter of the Association for Computational Linguistics
Session Student Session
Year 2012

This paper seeks to quantitatively evaluate the degree to which a number of popular met- rics provide overlapping information to parser designers. Two routine tasks are considered: optimizing a machine learning regularization parameter and selecting an optimal machine learning feature set. The main result is that the choice of evaluation metric used to optimize these problems (with one exception among popular metrics) has little effect on the solu- tion to the optimization.