Paper: Shared Task Proposal: Syntactic Paraphrase Ranking

ACL ID W12-1528
Title Shared Task Proposal: Syntactic Paraphrase Ranking
Venue International Conference on Natural Language Generation
Session Main Conference
Year 2012

We describe a new shared task on syntac- tic paraphrase ranking that is intended to run in conjunction with the main surface real- ization shared task. Taking advantage of the human judgments collected to evaluate the surface realizations produced by com- peting systems, the task is to automatically rank these realizations?viewed as syntactic paraphrases?in a way that agrees with the hu- man judgments as often as possible. The task is designed to appeal to developers of surface realization systems as well as machine transla- tion evaluation metrics: for surface realization systems, the task sidesteps the thorny issue of converting inputs to a common representation; for MT evaluation metrics, the task provides a challenging framework for advancing auto- matic evaluation, as many of the para...