Re-examining how utility and weighting functions get their shapes: a quasi-adversarial collaboration providing a new interpretation

Stewart, Reimers and Harris (2015, SRH hereafter) demonstrated that shapes of utility and probability weighting functions could be manipulated by adjusting the distributions of outcomes and probabilities on offer, as predicted by the theory of Decision by Sampling. So marked were these effects that, at face value, they profoundly challenge standard interpretations of preference theoretic models where such functions are supposed to reflect stable properties of individual risk preferences. Motivated by this challenge, we report an extensive replication exercise based on a series of experiments conducted as a quasi-adversarial collaboration across different labs and involving researchers from both economics and psychology. We replicate the SRH effect across multiple experiments involving changes in many design features; importantly, however, we find that the effect is also present in designs modified so that Decision by Sampling predicts no effect. While those results depend on model-based inferences, an alternative analysis using a model free comparison approach finds no evidence of patterns akin to the SRH effect. On the basis of simulation exercises, we demonstrate that the SRH effect may be a consequence of misspecification biases arising in parameter recovery exercises that fit imperfectly specified choice models to experimental data. Overall, our analysis casts the SRH effect in an entirely new light.