Demand Effects in Survey Experiments: An Empirical Assessment

Citation:

Mummolo, Jonathan, and Erik Peterson. “Demand Effects in Survey Experiments: An Empirical Assessment”. (Working Paper). Print.

Abstract:

 

Survey experiments are ubiquitous in the behavioral social sciences. A frequent critique of this type of study is that evidence which supports a researcher's expectations occurs due to Experimenter Demand Eects (EDEs): bias stemming from participants inferring the purpose of an experiment and responding so as to help conrm a researcher's hypothesis. In this paper, we argue that traditional survey experimental designs have several features that make them robust to EDEs. We then explicitly test for the presence of EDEs using a series of experiments which randomly assign participants to receive varying levels of information about each experiment's hypothesis. Replicating three widely used experimental designs, we nd that informing participants of an experiment's purpose has no detectable eect on observed treatment eects, or attenuates them. When informed of a study's purpose, participants do not appear to assist researchers, a nding with important implications for the design and interpretation of survey experiments.

 

Last updated on 09/26/2017