Romney Adams
Romney Adams is a Learning Specialist at DistillerSR, responsible for planning and delivering Instructor-Led training to customers on all aspects of the platform - including our AI Classifier capabilities. She also designs and develops Self-Paced courses housed in DistillerSR Learn, the company's customer-facing LMS. Prior to joining DistillerSR, Romney worked for many years as a Liaison Librarian at Monash University, one of Australia's largest research institutions - so is no stranger to Systematic Reviews!
Session
Background: Systematic reviews (SRs) can be retrieved in several ways. One approach is a search filter designed to retrieve SRs in bibliographic databases such as PubMed. Another utilizes machine learning (ML) to sort articles into two mutually exclusive classes in online platforms such as DistillerSR.
Objective: To test the performance of a binary ML classifier designed to identify SRs in DistillerSR against search filters designed to retrieve SRs in PubMed.
Methods: Umbrella reviews will be identified, included SRs will be extracted to create a reference set. Each SR in the reference set will be verified as indexed in PubMed. Two SR search filters will be tested: a narrow SR filter and a broad SR filter. A binary ML classifier developed by DistillerSR to identify SRs will be used, informed by the reference set and a seed of non SR articles. The number of SRs from the reference set retrieved by the two search filters, and the number of SRs from the reference set correctly identified as a SR by the ML classifier will each be recorded discretely.
Results: Relative recall, precision and F1 score will be calculated for each set. Recommendations will be made on when to apply a search filter or ML classifier to a search.
Conclusions: Different approaches to limiting search results to specific study designs can influence the overall search strategy. The objectives of the project and resource requirements need to be considered when deciding on the approach.