Abstract:

Researchers want high-quality data from their participants. As long as many researchers across the social sciences have been collecting large amounts of data from online convenience samples like MTurk and Prolific, some have expressed skepticism whether data from these sources are high-quality. Such skepticism stems in part from the fact that online survey compensation is often low (e.g., 10 cents per minute) and the fact that a concerning percentage of participants fail English language comprehension questions as well as attention checks. Despite plausible reasons to expect low-quality, researchers have replicated online the psychometric properties of well-known personality and individual differences instruments (e.g., Big Five, Adult Attachment) as well as classic experimental findings (e.g., framing and anchoring effects). Such findings suggest high-quality. But how? We wondered whether these data are of high quality in part because participants are simply motivated to thoughtfully and honestly respond to surveys for reasons other than money. Across four convenience samples, we assessed data quality, survey-taking motivation, and self-reported engagement in surveys. I will present and discuss the results during my talk.