Can Generative Llms Create Query Variants For Test Collections? An Exploratory Study · Awesome LLM Papers Contribute to LLM-Bible

Awesome LLM Papers

A curated collection of research papers on Large Language Models (LLMs), transformers, prompting, fine-tuning, and multi-modal systems. Maintained by Sean Moran.

Contact Sean Moran about this survey or website. Made with Jekyll and Hyde.

Can Generative Llms Create Query Variants For Test Collections? An Exploratory Study

Marwah Alaofi, Luke Gallagher, Mark Sanderson, Falk Scholer, Paul Thomas. SIGIR '23: The 46th International ACM SIGIR Conference on Research and Development in Information Retrieval 2025 – 23 citations

[Paper]    
Reinforcement Learning Fine-Tuning Evaluation

This paper explores the utility of a Large Language Model (LLM) to automatically generate queries and query variants from a description of an information need. Given a set of information needs described as backstories, we explore how similar the queries generated by the LLM are to those generated by humans. We quantify the similarity using different metrics and examine how the use of each set would contribute to document pooling when building test collections. Our results show potential in using LLMs to generate query variants. While they may not fully capture the wide variety of human-generated variants, they generate similar sets of relevant documents, reaching up to 71.1% overlap at a pool depth of 100.

Similar Work