Paper: Using Clustering to Improve Retrieval Evaluation without Relevance Judgments

ACL ID C10-2130
Title Using Clustering to Improve Retrieval Evaluation without Relevance Judgments
Venue International Conference on Computational Linguistics
Session Poster Session
Year 2010
Authors

Retrieval evaluation without relevance judgments is a hard but also very mean- ingful work. In this paper, we use clus- tering technique to improve the per- formance of judgment free retrieval evaluation. By using one system to rep- resent all the systems that are similar to it, we can largely reduce the negative ef- fect of similar retrieval results in Re- trieval evaluation. Experimental results demonstrated that our method outper- formed all the previous judgment free evaluation methods significantly. Its overall average performance outper- formed the best previous result by 20.5%. Besides, our work is a general framework that can be applied to any other judgment free evaluation method for performance improvement.