SIPS 2026 DC

How Much Do Unknown and Undisclosed Program Defaults Harm Reproducibility?
2026-06-08 , AUDITORIUM

Statistics programs use defaults. When a user does not specify every single option in their code, default choices are often chosen for them. These can be as simple as defaulting to Welch's t-test versus Student's t, or more complicated including default scaling in structural equation modeling or maximum likelihood procedure in mixed effects models. We are looking for people who are interested in the topic of computational reproducibility and familiar with any of the following (non exhaustive) statistical techniques:
t-test, ANOVA, Regression, SEM/Factor Analysis, MLM, Survival Analysis,
and know how to use one of the following (non-exhaustive) statistical software:
R, STATA, SPSS, MPlus, SAS
The goal is to use open datasets to run the same model using only program defaults in each of the different programs, highlighting if and when results differ.


Please classify your session as the theme it fits best in:: Logistics/Tools - Content related to creation or refinement of new tools, databases, and methods What is your end product?:

A paper highlighting if and when statistical defaults change results using the same data across programs.

How will the session's content foster diversity & inclusion (e.g., who will present, who will it serve), and how will it improve psychological science?:

Computational reproducibility is paramount to ensuring the reliability of published findings. Not everyone uses the same statistics program, and they never will. When trying to reproduce results across statistical programs, different decisions can be made based on the programs' defaults. As some programs are more inclusive than other programs (due to license fees etc.), we may highlight which programs' defaults cause more or fewer problems.

Please note any pre-requisite knowledge/expertise you will expect from attendees (i.e., is the session most appropriate for someone who already has experience with a topic?).:

Knowledge of running basic statistics.