Interpreting studies where the main finding replicates but not the control condition
We replicated Hsee (1998), which found that when evaluating choices jointly, people compare and judge the option higher on desirable attributes as better (“more is better”). However, when people evaluate options separately, they rely on contextual cues and reference points, sometimes resulting in evaluating the option with less as better (“less is better”). Results support the (surprising) “less is better” effect across all studies (N=403; Study 1 original d = .70, replication d = .99; Study 2 original d = .74, replication d = .32; Study 4 original d = .97, replication d = .76), with weaker support for the (obvious) “more is better” (Study 2 original d = .92, replication dz = .33; Study 4 original d = .37, replication dz = .09). I will discuss interpreting the meaning of a study when the “surprising” part replicates but the “obvious” part doesn’t. It suggests the purported mechanism behind the effect may not be supported.