Rosie Le Faive

Rosie Le Faive is the Metadata Librarian at the University of Prince Edward Island. They have 13 years' experience writing code for and wrangling Drupal, including with the Islandora project. Their interests include knowledge justice, information architecture, and UX design. When not working with computers, they enjoy spinning wool and hanging out with cats. They live and work in Mi’kma’ki.


Session

06-04
14:30
20min
Assessing the performance of the National Library of Medicine’s Medical Text Indexer - neXt-generation (MTIX) MEDLINE indexing algorithm
Alexandre Amar-Zifkin, Tara Landry, Rosie Le Faive

Introduction
Since April 2022, indexing of MEDLINE records is performed predominantly by algorithm, with occasional human intervention. Recent research has raised concerns about indexing algorithms’ capacity to accurately identify meaningful elements of publications. This identification is a necessary precondition to communicating them to searchers through indexing. Concerns have included the risk of the algorithm overlooking one or more relevant concepts, not making use of appropriately precise controlled vocabulary, or assigning incorrect terms due to rhetorical or technical language. The indexing algorithm from 2022-2024 (MTIA) was a rule-based system with exceptions added over time; a machine learning model (MTIX) was implemented in 2024. NLM Technical Bulletins have noted general improvements in MTIX compared to MTIA. Recent research has explored particularities of evocative language and found improvements in discerning human from nonhuman-animal subjects.
METHODS
This work will replicate and improve on a previous project that assessed overall MTIA indexing performance. Our study will assess MTIX performance by reviewing a large, recent and random sample of MTIX-indexed records to determine whether their main concepts are adequately represented.
Using a web-form displaying the journal, title and abstract of a record, our screeners will identify key concepts that, per their searching experience, would be used to retrieve it and similar records. After establishing agreement between screeners, we will compare the consensus-concepts to the indexing applied by MTIX. We will compare these findings to previous research, noting emergent trends or issues.

RESULTS & CONCLUSION
We intend to make our web-form freely available online. Other results are forthcoming.

AI
Borduas-Krieghoff2