Introduction:
Artificial intelligence is impacting nearly every stage of the evidence synthesis process. Generating appropriate subject headings for a literature search is an area where GenAI models often fall short, but these tools are constantly changing. This longitudinal study examines GenAI models’ performance when suggesting subject terms for comprehensive search strategies.

Methods:
We are testing free and paid versions of ChatGPT, Gemini, and Claude using prompts designed to generate MeSH, EMTREE, CINAHL, and APA PsycINFO subject terms. Judgement sampling was used to gather published reviews and related search histories from a variety of health sciences topics. Included reviews were co-authored by a librarian, used a peer-reviewed search, provided at least a MEDLINE search, and were only accessible behind a paywall. An additional original research question and peer reviewed search was devised by the authors. These searches act as gold standard comparators for the subject terms generated by the models. Prompts are run every two weeks. The alignment of GenAI model-suggested subject terms with the corresponding gold standard searches, and the occurrence of hallucinations will be analysed.

Results: Preliminary results will be available in mid-2026.

Discussion: Our results will provide the first longitudinal analysis on the performance of large-language models to suggest subject terms for evidence synthesis research. These results will help search experts and researchers to make informed decisions on whether to integrate GenAI tools into their search strategy development process.