Sarah Neil-Sztramko


Interventions

04/06
11:00
20minutes
Artificial Intelligence-Supported Screening: Does it Impact Evidence Certainty and Time to Complete a Rapid Review?
Sarah Neil-Sztramko

Introduction: The National Collaborating Centre for Methods and Tools (NCCMT) established its Rapid Evidence Service (RES) to support evidence-informed public health by conducting timely, rapid reviews on priority topics. Artificial intelligence (AI) screening features offer the potential to automate and expedite the review process, while reducing unintended human biases or errors, but there is limited evidence to quantify this impact. This study aims to evaluate how AI compares to manual screening with respect to missed studies, impact on overall review findings, and time to complete.

Methods: Two AI features, Re-Rank and Check Screening Errors, were compared to manual dual screening during title and abstract screening (DistillerSR, v2.35). As screening occurred in each review, project clones were made at likelihood thresholds of 60-95%, AI screened remaining references, and potential false excludes were identified. These AI-screened results were compared with manual screening in the original projects to identify how many studies would have been missed at each threshold. The impact of omitting missed studies on the review’s key findings was assessed. Finally, time spent screening was tracked across reviews.

Results: Six rapid reviews were conducted during the study period. In preliminary data analysis, AI correctly excluded up to 60% (2600 out of 4100 studies) at a prediction threshold as low as 80% in one review; time spent screening was 47 hours.

Discussion: AI is a promising support tool for improving screening efficiency and accuracy. Additional study is needed to understand how AI can be most appropriately integrated into rapid review methods.

AI
2314
04/06
14:30
20minutes
Promoting the use of Synthesized Evidence in Public Health Decision Making: A Rapid Evidence Service and Repository of Public Health Evidence Syntheses
Sarah Neil-Sztramko

Introduction: Evidence syntheses can be used to inform public health policy and programming decisions, however, organizations may lack the resources and expertise to develop timely, high-quality syntheses. The National Collaborating Centre for Methods and Tools (NCCMT) launched a Rapid Evidence Service (RES) to answer priority public health questions through rapid reviews. The NCCMT also established the Repository of Public Health Evidence Syntheses (the Repository) to minimize duplication of efforts, facilitate collaboration, and ensure public health practitioners have access to relevant evidence syntheses.

Description: The RES process involves collaboratively developing a research question, conducting a comprehensive search, and critically appraising and synthesizing the findings into an actionable report. Each RES report is included in the Repository and made available to the public health community. Other organizations also contribute their syntheses to the Repository. Users can search the Repository for existing syntheses and adapt the findings for use in their own communities or connect with authors to collaboratively address public health issues.

Outcomes: Since launching in 2020, the RES and Repository have focused on many timely public health topics, such as COVID-19, climate change, and health equity. The RES has collaborated with over 20 organizations to complete more than 100 reviews and review updates on 61 topics. The Repository hosts over 600 submissions from over 75 organizations and has been accessed over 30,000 times.

Discussion: The RES and Repository facilitate access to relevant, high-quality evidence syntheses. These services promote the use of synthesized evidence, encourage interorganizational collaboration and ease knowledge exchange in public health decision making.

Knowledge Synthesis
2314