The Combination and Evaluation of Query Performance Prediction Methods


Share/Save/Bookmark

Hauff, C. and Azzopardi, L. and Hiemstra, D. (2009) The Combination and Evaluation of Query Performance Prediction Methods. In: Proceedings of the 31st European Conference on IR Research on Advances in Information Retrieval, 6-9 April 2009, Toulouse, France (pp. pp. 301-312).

[img] PDF
Restricted to UT campus only
: Request a copy
242kB
Abstract:In this paper, we examine a number of newly applied methods for combining pre-retrieval query performance predictors in order to obtain a better prediction of the query's performance. However, in order to adequately and appropriately compare such techniques, we critically examine the current evaluation methodology and show how using linear correlation coefficients (i) do not provide an intuitive measure indicative of a method's quality, (ii) can provide a misleading indication of performance, and (iii) overstate the performance of combined methods. To address this, we extend the current evaluation methodology to include cross validation, report a more intuitive and descriptive statistic, and apply statistical testing to determine significant differences. During the course of a comprehensive empirical study over several TREC collections, we evaluate nineteen pre-retrieval predictors and three combination methods.
Item Type:Conference or Workshop Item
Faculty:
Electrical Engineering, Mathematics and Computer Science (EEMCS)
Research Group:
Link to this item:http://purl.utwente.nl/publications/67852
Official URL:http://dx.doi.org/10.1007/978-3-642-00958-7_28
Export this item as:BibTeX
EndNote
HTML Citation
Reference Manager

 

Repository Staff Only: item control page