Paper: Incorporating Non-Local Information Into Information Extraction Systems By Gibbs Sampling

ACL ID P05-1045
Title Incorporating Non-Local Information Into Information Extraction Systems By Gibbs Sampling
Venue Annual Meeting of the Association of Computational Linguistics
Session Main Conference
Year 2005
Authors

Most current statistical natural language process- ing models use only local features so as to permit dynamic programming in inference, but this makes them unable to fully account for the long distance structure that is prevalent in language use. We show how to solve this dilemma with Gibbs sam- pling, a simple Monte Carlo method used to per- form approximate inference in factored probabilis- tic models. By using simulated annealing in place of Viterbi decoding in sequence models such as HMMs, CMMs, and CRFs, it is possible to incorpo- rate non-local structure while preserving tractable inference. We use this technique to augment an existing CRF-based information extraction system with long-distance dependency models, enforcing label consistency and extraction template consis- tency constr...