Poster: Still a filtering failure? Automated indexing using MTIX versus MTIA and its impact on human study filtering for knowledge synthesis
Introduction: The search filter ‘exp animals/ not humans.sh’ is a well-established method in knowledge synthesis used to exclude non-human studies in Ovid Medline. We previously reported on the impact of the Medical Text Indexer-Auto (MTIA) algorithm for automated assignment of MeSH terms on the utility of this filter for knowledge synthesis projects. We sought to update our reporting to account for the 2024 implementation of the new Medical Text Indexer-NeXt Generation (MTIX) algorithm, which uses a machine-learning model for MeSH term assignment.
Methods: As in the previous study, we conducted a search in Ovid Medline using the Cochrane Highly Sensitive Search Strategy. We isolated the results indexed by the automated method and specifically excluded by the non-human-studies filter in the timeframe since MTIX was implemented. We screened these results using Covidence to identify human studies.
Results: The sample demonstrated a significant improvement over our assessment of MTIA: only 1% (25/2285) of studies screened were inappropriately excluded human studies - compared to 4.2% in the MTIA assessment - and none of these were in a clinical context. Records describing both animal and human studies continue to be a common source of inappropriate exclusion.
Discussion: Our findings suggest that the filter is much less likely to inappropriately exclude human studies indexed by MTIX (records indexed beginning April 2024) than MTIA (studies indexed between 2019 and April 2024). However, we still recommend caution with the use of the human studies filter, especially for records indexed between 2019-2024.