Paper: PEM: A Paraphrase Evaluation Metric Exploiting Parallel Texts

ACL ID D10-1090
Title PEM: A Paraphrase Evaluation Metric Exploiting Parallel Texts
Venue Conference on Empirical Methods in Natural Language Processing
Session Main Conference
Year 2010
Authors

We present PEM, the first fully automatic met- ric to evaluate the quality of paraphrases, and consequently, that of paraphrase generation systems. Our metric is based on three crite- ria: adequacy, fluency, and lexical dissimilar- ity. The key component in our metric is a ro- bust and shallow semantic similarity measure based on pivot language N-grams that allows us to approximate adequacy independently of lexical similarity. Human evaluation shows that PEM achieves high correlation with hu- man judgments.