MPSDS / JPSM Seminar Series
MPSDS M3 Series: Mastery, Methodology, Meetups

In person, room 1070 Institute for Social Research, and via Zoom.
The Zoom call will be locked 10 minutes after the start of the presentation.

From Survey to SurvAI: The Promises and Precautions of AI for Survey Research

Large language models (LLMs) are rapidly transforming many professional domains, including survey research. Eloundou et al. (2024) rank survey research among the most highly exposed occupations to LLM-driven automation, raising both opportunities and challenges for practitioners. While survey science has a rich tradition of adopting technological tools for tasks like data collection, analysis, and instrument design, the unique affordances and risks associated with LLMs call for a structured examination.

This paper presents findings from a systematic literature review of empirical and theoretical work at the intersection of LLMs and survey research. Specifically, we sought to synthesize examples of how LLMs are being applied across three broad phases of the survey research pipeline including: pre-data collection, data collection and post-data collection. Methodologically, this review identifies uneven distribution of LLM application across the survey pipeline. While pre-data collection stages (e.g., item writing, translation) are well explored, core practices like live interviewing, recruitment, and cross-lingual adaptation remain under-investigated. Additionally, few studies assess LLMs systematically across multiple populations, languages, or survey topics. In this presentation we will highlight not just the breadth of current use cases, but also the methodological and ethical considerations that must accompany them noting examples that are both promising as well as precautionary.

Trent D. Buskirk, Ph.D. has recently joined the new School of Data Science at Old Dominion University. Prior to this appointment, Trent was the Novak Family Distinguished Professor of Data Science and outgoing Chair of the Applied Statistics and Operations Research Department at Bowling Green State University. Dr. Buskirk is a Fellow of the American Statistical Association and his research is positioned at the intersection of survey science, data science, computational social science, and human–AI interaction. His specific research interests include Schema-Driven LLM-Based Inference, big data quality, recruitment methods through social media, the use of big data and machine learning methods for health, social and survey science design and analysis, mobile and smartphone survey designs and in methods for calibrating and weighting samples and fairness in AI models and interpretable ML methods. Trent has also been involved in various professional organizations serving as the President of the Midwest Association for Public Opinion Research in 2016, the Conference Chair for AAPOR in 2018 and a member of the scientific committee for the BigSurv series of conferences since 2018. Trent as also served as an Associate Editor (Methods) for the Journal of Survey Statistics and Methodology. Dr. Buskirk is currently serving on the AAPOR Responsible Integration of AI in Survey Research task force. When Trent is not geeking out over data science or survey research, he’s likely out playing a competitive game of Pickleball!