Edinburgh Napier University

  Query performance prediction (QPP) aims to predict the success and failure of a search engine on a collection of queries and documents. State of the art predictors can enable this prediction with a degree of accuracy; however, it is far from being perfect. Existing studies have mainly observed QPP is a difficult task but yet have lacked in-depth qualitative analysis. In this paper, we analyze QPP from the perspective of predicting the accuracy of query performance. Our working hypothesis is that certain queries lend themselves more easy to prediction while others pose greater challenges. Moreover, by focusing on outliers, we can pinpoint queries that are particularly difficult to predict. To achieve this, we consider multivariate outlier detection. Our results show the effectiveness of this approach in identifying queries for which QPP struggles to provide accurate predictions. Furthermore, we show that by excluding these difficult to predict queries, the overall accuracy of QPP is substantially improved.