Simulating the Future of Concept-Based Video Retrieval under Improved Detector Performance


Aly, R.B.N. and Hiemstra, D. and Jong de, F.M.G. and Apers, P.M.G. (2012) Simulating the Future of Concept-Based Video Retrieval under Improved Detector Performance. Multimedia Tools and Applications, 60 (1). pp. 203-231. ISSN 1380-7501

PDF (Publisher's version)
Abstract:In this paper we address the following important questions for concept-based video retrieval: (1) What is the impact of detector performance on the performance of concept-based retrieval engines, and (2) will these engines be applicable to real-life search tasks if detector performance improves in the future? We use Monte Carlo simulations to answer these questions. To generate the simulation input, we propose to use a probabilistic model of two Gaussians for the confidence scores that concept detectors emit. Modifying the model's parameters affects the detector performance and the search performance. We study the relation between these two performances on two video collections. For detectors with similar discriminative power and a concept vocabulary of around 100 concepts, the simulation reveals that in order to achieve a search performance of 0.20 mean average precision (MAP) -- which is considered sufficient performance for real-life applications -- one needs detectors with at least 0.60 MAP. We also find that, given our simulation model and low detector performance, MAP is not always a good evaluation measure for concept detectors since it is not strongly correlated with the search performance.
Item Type:Article
Copyright:© The Authors
Electrical Engineering, Mathematics and Computer Science (EEMCS)
Research Group:
Link to this item:
Official URL:
Export this item as:BibTeX
HTML Citation
Reference Manager


Repository Staff Only: item control page

Metis ID: 279155