Socratic Questionnaires

When experimental participants are given the chance to reflect and revise their initial judgments in a dynamic conversational context, do their responses to philosophical scenarios differ from responses to those same scenarios presented in a traditional static survey? In three experiments we find no consistent evidence that responses differ in these different formats. This aligns with recent findings that manipulations of reflectiveness have no effect on participants’ judgments about philosophical scenarios. Although we did not find a consistent quantitative effect of format (conversation vs. static survey), conversational experiments still provide qualitative insights into debates about how participants are understanding (or misunderstanding) the scenarios they read in experimental studies, and whether they are replacing difficult questions with questions that are more easily answered. We argue that conversational experiments—“Socratic questionnaires”—help show what is going on “under the hood” of traditional survey designs in the experimental investigation of philosophical questions.