Paper: Feature Selection for Fluency Ranking

ACL ID W10-4216
Title Feature Selection for Fluency Ranking
Venue International Conference on Natural Language Generation
Session Main Conference
Year 2010

Fluency rankers are used in modern sentence generation systems to pick sentences that are not just grammatical, but also fluent. It has been shown that feature-based models, such as maximum entropy models, work well for this task. Since maximum entropy models allow for in- corporation of arbitrary real-valued features, it is often attractive to create very general feature templates, that create a huge num- ber of features. To select the most discrim- inative features, feature selection can be ap- plied. In this paper we compare three fea- ture selection methods: frequency-based se- lection, a generalization of maximum entropy feature selection for ranking tasks with real- valued features, and a new selection method based on feature value correlation. We show that the often-used frequency-b...