Paper: Breaking Out of Local Optima with Count Transforms and Model Recombination: A Study in Grammar Induction

ACL ID D13-1204
Title Breaking Out of Local Optima with Count Transforms and Model Recombination: A Study in Grammar Induction
Venue Conference on Empirical Methods in Natural Language Processing
Session Main Conference
Year 2013
Authors

Many statistical learning problems in NLP call for local model search methods. But accu- racy tends to suffer with current techniques, which often explore either too narrowly or too broadly: hill-climbers can get stuck in local optima, whereas samplers may be inefficient. We propose to arrange individual local opti- mizers into organized networks. Our building blocks are operators of two types: (i) trans- form, which suggests new places to search, via non-random restarts from already-found local optima; and (ii) join, which merges candidate solutions to find better optima. Experiments on grammar induction show that pursuing dif- ferent transforms (e.g., discarding parts of a learned model or ignoring portions of train- ing data) results in improvements. Groups of locally-optimal solutions ...