Paper: Determining the Unithood of Word Sequences Using a Probabilistic Approach

ACL ID I08-1014
Title Determining the Unithood of Word Sequences Using a Probabilistic Approach
Venue International Joint Conference on Natural Language Processing
Session Main Conference
Year 2008
Authors

Most research related to unithood were con- ducted as part of a larger effort for the deter- mination of termhood. Consequently, nov- elties are rare in this small sub-field of term extraction. In addition, existing work were mostly empirically motivated and derived. We propose a new probabilistically-derived measure, independent of any influences of termhood, that provides dedicated measures to gather linguistic evidence from parsed text and statistical evidence from Google search engine for the measurement of unit- hood. Our comparative study using 1,825 test cases against an existing empirically- derived function revealed an improvement in terms of precision, recall and accuracy.