Query Performance Prediction: Evaluation Contrasted with Effectiveness


Hauff, Claudia and Azzopardi, Leif and Hiemstra, Djoerd and Jong, Franciska de (2010) Query Performance Prediction: Evaluation Contrasted with Effectiveness. In: Proceedings of the 32nd European Conference on IR Research, 28-31 March 2010, Milton Keynes, United Kingdom (pp. pp. 204-216).

[img] PDF
Restricted to UT campus only
: Request a copy
Abstract:Query performance predictors are commonly evaluated by reporting correlation coefficients to denote how well the methods perform at predicting the retrieval performance of a set of queries. Despite the amount of research dedicated to this area, one aspect remains neglected: how strong does the correlation need to be in order to realize an improvement in retrieval effectiveness in an operational setting? We address this issue in the context of two settings: Selective Query Expansion and Meta-Search. In an empirical study, we control the quality of a predictor in order to examine how the strength of the correlation achieved, affects the effectiveness of an adaptive retrieval system. The results of this study show that many existing predictors fail to achieve a correlation strong enough to reliably improve the retrieval effectiveness in the Selective Query Expansion as well as the Meta-Search setting.
Item Type:Conference or Workshop Item
Copyright:© 2010 Springer
Electrical Engineering, Mathematics and Computer Science (EEMCS)
Research Group:
Link to this item:http://purl.utwente.nl/publications/70843
Official URL:https://doi.org/10.1007/978-3-642-12275-0_20
Export this item as:BibTeX
HTML Citation
Reference Manager


Repository Staff Only: item control page