Intelligence Is Not Wisdom, with Kevin Collins

Farrah is joined by Kevin Collins, co-founder and Chief Research Officer at Survey 160, about the applicability of AI in survey methodologies, the perils of synthetic sample, and the importance of respondent preferences when it comes to survey mode and the respondent experience. We discussed Kevin's findings in some early explorations of available tools about the impact on data quality, the potential role of AI in coding open-ended survey responses, and where there might be opportunity for AI tools to support but not replace human expertise.

We also explored an issue we both have some opinions about - the concept of 'popularism' in political campaigning, how it intersects with different styles of persuasion, and how it influences other decision-making in how campaigns roll out messaging.

Our Guest

Kevin Collins is co-founder and Chief Research Office at Survey 160, a polling firm working to make SMS-based survey collection a staple mode of research to help fuel progressive campaigns. Survey 160 works with high-profile political campaigns at both the national and state level, as well as advocacy organizations and other groups working for a brighter future. You can connect with him on Bluesky @kwcollins.bsky.social or through their website at https://www.survey160.com/

Read their work on AI in survey methods here: The Limits of Simulation in Public Opinion Research

Other Sources Mentioned

• Out of One, Many: Using Language Models to Simulate Human Samples, by Argyle et al
• Predicting Results of Social Science Experiments Using Large Language Models, by Hewitt et al
• Synthetic Replacements for Human Survey Data? The Perils of Large Language Models, by Bisbee et al
• Do AIs know what the most important issue is? Using language models to code open-text social survey responses at scale, by Mellon et al
• Context Length Alone Hurts LLM Performance Despite Perfect Retrieval, by Du et al
• Does AI Actually Boost Developer Productivity? (100k Devs Study) - Yegor Denisov-Blanch, Stanford [video]
• Does accommodation work? Mainstream party strategies and the success of radical right parties, by Krause et al
• The Nature and Origins of Mass Opinion, by John R. Zaller [book]
• Consuming Cross-Cutting Media Causes Learning and Moderates Attitudes: A Field Experiment with Fox News Viewers, by David E. Broockman and Joshua L. Kalla

Stay in Touch

📬 Subscribe to our newsletter at crosstabspodcast.com for new episodes, insights, and behind-the-scenes content.

📹 Watch video episodes on YouTube @CrosstabsPodcast

💬 Follow us on BlueSky: @crosstabspod.bsky.social and @farrahbostic.bsky.social

📍 Produced by The Difference Engine