04/06/2025 –, 2314
Langue: English
Introduction
There has been a recent explosion of AI-powered research assistants designed to find and summarize scientific literature. These nascent tools require evaluation to foster critical thinking about their use and to inform improvements. This study compares AI-generated syntheses of the literature to the conclusions of Cochrane systematic reviews of the same clinical questions.
Methods
Cochrane systematic reviews (SRs) of exercise with published March 2024 to 2025 were selected. Four AI-powered research assistants (AI tools) and Copilot were queried using the SR research questions. The following data from the SRs and the AI tool summaries was extracted independently by the two researchers, with disagreements resolved through consensus: the number, study design, and citations of included articles, assessment of quality or risk of bias (if available), and conclusions regarding the intervention(s). Agreement between SRs and AI tools will be assessed by coding conclusions as agreeing, disagreeing, or unclear. Descriptive statistics will be used to compare the literature cite for the conclusion or summary.
Results - Data analysis is underway.
Discussion - This analysis will inform our understanding of the accuracy, reliability and reproducibility of these AI research assistant tools.