oHA4: Moving Past Prestige: Developing an Umbrella Review of Citation Metrics Relationship to Research Quality
Citation metrics are still relied on when making science management decisions: from hiring and rewarding individual scientist, financing research groups, to budgeting for departments and whole universities. This goes on despite many primary studies showing the invalidity of citation metrics as indicators of scientific rigor or quality. Individual studies ranked on their citation count, and journals raking on their impact according to various metrics has been found to be negatively related to the statistical power of reported research ("N-pact factor") , research replicability , and instead positively related to the presence statistical reporting errors.
To increase the velocity of abandoning these metrics in science management decisions, we need a definitive, synthesized evidence base. This hackathon aims to lay the groundwork for a comprehensive Umbrella Review or large-scale meta-analysis assessing the validity of journal metrics as indicators of research quality, rigor, or impact.