ACM Transactions on Information Systems (TOIS) seeks previously unpublished papers on information retrieval (such as search engines, recommender systems, conversational search agents) that contain:

    new principled information retrieval models or algorithms with sound empirical validation;
    observational, experimental and/or theoretical studies yielding new insights into information retrieval or information seeking;
    accounts of applications of existing information retrieval techniques that shed light on the strengths and weaknesses of the techniques;
    formalization of new information retrieval or information seeking tasks and of methods for evaluating the performance on those tasks;
    development of content (text, image, speech, video, etc.) analysis methods to support information retrieval and information seeking;
    development of computational models of user information preferences and interaction behaviors;
    creation and analysis of evaluation methodologies for information retrieval and information seeking; or
    exceptionally well-written surveys of existing work that propose a significant synthesis.

The information retrieval scope of ACM Transactions on Information Systems (TOIS) appeals to industry practitioners for its wealth of creative ideas, and to academic researchers for its descriptions of their colleagues' work.

Special Issue on Query Performance Prediction Towards Novel Information Retrieval Paradigms
Submission Date: 2025-03-15

Guest Editors:
• Dr. Suchana Datta, University College Dublin, Ireland, suchana.datta@ucdconnect.ie
• Dr. Guglielmo Faggioli, University of Padua, Italy, guglielmo.faggioli@unipd.it
• Prof. Nicola Ferro, University of Padua, Italy, ferro@dei.unipd.it
• Dr. Debasis Ganguly, University of Glasgow, United Kingdom, debasis.ganguly@glasgow.ac.uk
• Prof. Iadh Ounis, University of Glasgow, United Kingdom, iadh.ounis@glasgow.ac.uk

This Special Issue focuses on Query Performance Prediction (QPP) with respect to recent advances in Information
Retrieval (IR).

QPP is a branch of IR evaluation: it is defined as the task of assessing or predicting the performance of a query
without human-made relevance judgements.

The focus of the special issue will be on three major topics concerning QPP:
• The development of novel QPP models that employ recent neural state-of-the-art solutions, such as
Large Language Models (LLMs) and semantic representations.
• The application of QPP models to novel IR tasks, such as conversational search, fairness-oriented tasks,
multimedia and multimodal retrieval, and Retrieval Augmented Generation (RAG).
• The evaluation of QPP methods performance.

In particular, classical QPP models have been developed focusing on IR systems based on exact matching (i.e.,
systems that consider the presence of a query term in the document as the only relevance signal). Advances in IR,
especially linked to the development of neural IR models and the recent extensive usage of LLMs, have also
highlighted the importance of semantic signals in producing good retrieval results. Therefore, this special issue aims
to attract works, which employ techniques that make use of modern neural IR approaches for the QPP task, thereby
aligning the QPP models to today’s state-of-the-art IR systems.

Secondly, QPP has been traditionally applied to ad-hoc retrieval. In ad-hoc retrieval, the user issues a – typically
short – natural language query and obtains a set of documents in response. Modern IR systems tend to be more
complex with multiple interactions between the user and the system, such as in the case of conversational search,
might include additional modules, such as in the case of RAG, and do not necessarily operate on text, such as in the
case of multimodal or multimodal IR. Therefore, this special issue aims at capturing works that operationalise QPP
for IR in novel domains and tasks.

Finally, evaluating QPP models has always been particularly challenging as we need large amounts of annotated
data and fine-grained evaluation measures. Furthermore, the advent of new IR paradigms, some relying on a
completely different definition of “performance”, has further exacerbated the problem. Thus, this special issue aims
to attract works that propose an advancement over the current state-of-the-art for what concerns QPP evaluation.

Topics
We welcome submissions on the following topics, including but not limited to:
• Application of QPP to Neural Information Retrieval Systems
• Usage of QPP for modern tasks, including, but not limited to, conversational search, fairness, RAG,
multimodal retrieval
• Usage of Large Language Models for QPP
• QPPs based on non-lexical (e.g., semantic, multimodal) signals
• Supervised QPP
• Simulation and construction of evaluation collections with Large Language Models
• QPP evaluation measures
• Development of QPP evaluation collection
• Performance Prediction in neighboring areas including NLP and Recommender Systems
• Theory underneath QPP
• Applications of QPP for downstream tasks, e.g., selective application of second-stage ranking or
relevance feedback.
• Explainability of QPP models and QPP models for explainability

Important Dates
• Submissions deadline: March 15, 2025
• First-round review decisions: May 15, 2025
• Deadline for revision submissions: July 15, 2025
• Notification of final decisions: September 15, 2025
• Tentative publication: Late 2025

Submission Information

Authors can submit their manuscripts via https://dl.acm.org/journal/tois. Submissions to this special issue will
follow the regular TOIS submission guidelines (https://dl.acm.org/journal/tois/author-guidelines). Submissions
must be accompanied by a cover letter containing all of the following: (1) Confirm that the paper is not currently
under submission at another journal or conference. (2) Confirm that the paper is substantially different from any
previously published work. (3) Confirm that none of the co-authors is a Guest Editor for this special session. (4)
Disclose possible conflicts of interest with Guests Editors. The review process will be single-blind. Strict policies will
be followed for plagiarism, submission confidentiality, reviewer anonymity, prior and concurrent paper submission
based on the guidelines.

Papers with a “Major Revision” decision should be resubmitted within three months, and with a “Minor Revision”
decision should be resubmitted within one month. Revised submissions must be accompanied with a detailed
response to reviewers explaining what revisions were implemented. The editors will conduct a second-round review
process and give the decision (accept or reject or need further revision) in one month.

For questions and further information, please contact Dr. Guglielmo Faggioli at guglielmo.faggioli@unipd.it.