SIPS 2026 DC

Evidence synthesis, particularly meta-analysis, is treated as a ‘gold standard’ of scientific evidence. It is relied upon to inform clinical treatments, policy decisions, and other real-world applications of research. However, a meta-analysis is only as credible as the (least credible) primary studies that it includes. We argue that meta-analysis is a prime opportunity for post-publication critique, and indeed that such critical appraisal is necessary for valid meta-analyses. In the field of psychology, the replication crisis demonstrated that even studies published in influential journals are more likely to contain false positives, questionable research practices, and fraud than previously assumed. It is therefore crucial that primary studies are critically evaluated before they are included in a meta-analysis. We present an in-progress project evaluating the quality of primary studies included in published meta-analyses.