Paper: Regularized Structured Perceptron: A Case Study on Chinese Word Segmentation, POS Tagging and Parsing

ACL ID E14-1018
Title Regularized Structured Perceptron: A Case Study on Chinese Word Segmentation, POS Tagging and Parsing
Venue Annual Meeting of The European Chapter of The Association of Computational Linguistics
Session Main Conference
Year 2014
Authors

Structured perceptron becomes popular for various NLP tasks such as tagging and parsing. Practical studies on NLP did not pay much attention to its regularization. In this paper, we study three simple but effec- tive task-independent regularization meth- ods: (1) one is to average weights of dif- ferent trained models to reduce the bias caused by the specific order of the train- ing examples; (2) one is to add penalty term to the loss function; (3) and one is to randomly corrupt the data flow during training which is called dropout in the neu- ral network. Experiments are conducted on three NLP tasks, namely Chinese word segmentation, part-of-speech tagging and dependency parsing. Applying proper reg- ularization methods or their combinations, the error reductions with respect to the av- e...