Edinburgh Napier University

  Introduction. This paper presents the results of a study investigating the information literacy and search skills of young people in Scotland.
Method. The participants, secondary school pupils between the ages of 13 and 14 (n=57), completed two out of four different search tasks from the TREC HARD collection, for which the correct answers (i.e. relevant documents) were known. Their interactions with the search system were logged and information about their own perceptions of the task were collected through pre- and post-task questionnaires.
Analysis. The log data from the search system was analysed using the R statistical software package to understand the performance and behaviour of the participants when conducting the search tasks.
Findings. While we identified some evidence that information literacy and search skills were being employed, overall performance was low with participants often unable to produce successful queries and/or unable to identify relevant documents, even when some were present in the results. Despite assessing their own performance as being good, the pupils struggled to formulate good quality queries to assess documents for relevance, frequently selecting non-relevant sources.
Conclusion. Search performance and ability to identify relevant information was generally poor, a fact that participants themselves were frequently unable to recognise. The results also suggest a reliance on complex search assistance tools (such as spell checking and query suggestions), which are common features of major search engines, but not of smaller systems, which pupils are also likely to have to use. Despite the pupils having been giving some information literacy training in the previous year, the results suggest that more needs to be done to help school pupils in searching for and assessing relevant source documents.