AI Perspectives
We’ve been fielding a lot of questions from our clients about the practical role of generative Al and LLM tools like ChatGPT in qualitative market research. Which prompted us to create a series of Thinkpieces exploring common topics cropping up among our peers.
Titled “Al Perspectives,” these pieces feature diverse perspectives from three members of the Thinkpiece team: CEO Bonnie Dibling (self-proclaimed Al newbie), Chris Dethloff (Technology Moderator and Super-Smart Guy), and COO/Director of Technology Research John Dibling (who studied Al in college way back when).
Because Al is always evolving, there are no definitive answers to Al-related questions. Our goal, therefore, is to help us — and you — think differently and deliberately about how we use these tools in our daily work.
Have an Al-related question you’d like us to explore? Ask away.
ChatGPT can be a tool to help jump-start, refine, and improve a discussion guide, but it may not be worth the time and effort needed to get a useable guide. You still need a qual researcher (who is also a person) to write a guide that sparks a meaningful, insightful human conversation.
I’ve tried using ChatGPT a couple of times to help me with discussion guides, with disappointing results. As someone who is not a prompt engineer or terrifically versed in Al, the problem may have been with my prompts. On my first go-around, l provided ChatGPT with the objectives of the research, what l wanted to know, and the time frame for the discussion (60 minutes), then told it to give me the discussion guide.
The result was exactly what I was expecting.
ChatGPT’s wording was very stilted and clearly not human, and the questions had no real flow. It sounded like a robot. So, I tried changing my tactic and prompted with, “Here’s the one thing I need to find out about, how should l ask specifically about this particular topic in a friendly, neutral way?” And then ChatGPT went to the other extreme, with questions that started with phrases like “Hey, dude!” I tried again a few more times with similar outcomes, and then bailed on it.
In all honesty, I don’t want to have to learn how to be a prompt expert. And I’m sure l’m not alone in that. So I don’t think Al is ever going to replace a discussion guide for an actual conversation. The other issue I had to contend with was feeding ChatGPT the prompts in a way that was neutered enough to protect my client’s information.
All in all, using ChatGPT was causing more trouble than it was worth. By the time I got through all the trial and error, I could’ve just written the discussion guide myself.
While I don’t think Al will ever have a role in discussion guides, using it for surveys may be a completely different thing. For quantitative questions that are close-ended and very pinpointed, ChatGPT could probably generate a survey in 30 seconds instead of 30 days a human might spend on it. But for qualitative work, I don’t see ChatGPT getting there simply because it doesn’t know how to have a human conversation. Yet.
Yes, you could technically have a generative Al or an LLM like ChatGPT write an entire discussion guide. How performant that will be, I don’t know. Do I use ChatGPT to write discussion guides? No. You simply can’t rely on Al to do 100% of your work. But Al can fine-tune your work and be used as a jumping-off point. It can aid you in your creation of a discussion guide.
I would never let ChatGPT create an entire discussion guide end-to-end from scratch. But you could approach ChatGPT with the core of the question you want to ask, and leverage it to help you with wordsmithing, creating effective.
questions, and providing the building blocks to help put the guide together. And the more specific the input and direction you give a tool like ChatGPT, the better it will be able to help you.
That being said, is Al really going to save you time considering that you still have to put in the work and effort to feed it all that specific guidance — for instance, creating a complete discussion guide tr share with ChatGPT as an example.
Given that time investment, I might as well just create the entire discussion guide myself.
There is a role for Al in writing discussion guides, but there are also a lot of caveats and issues to be aware of and careful about. Privacy is first and foremost among those. The work that we do is highly confidential, and in some cases revealing confidential information could be business-ending. It’s super critical that we protect our clients’ privacy, not only in terms of their name but also the topic we’ll be discussing in our studies.
To do that, we have to remain ambiguous, which does limit the ability to use a generative Al tool like ChatGPT for discussion guides. If you can figure out your way past the privacy concerns and avoid revealing who your client is and what
they’re working on, I think ChatGPT as it is today can be used for small, granular tasks in writing discussion guides — not for large-scale “go write a discussion guide” type of work. The capabilities just aren’t there. You can try it, and ChatGPT will spit something out that might look convincing. But it probably won’t be a very good discussion guide.
The small, granular tasks that ChatGPT could be useful for, however, are requests like, “Help me to reframe this question so that it’s open-ended and unbiased.” Or, “Help me analyze the flow of my discussion guide.” Or, “Help me find the gaps in the guide I wrote based on objectives a, b, and c.” Or, “Help me find areas where I’m being redundant.” So, ChatGPT basically becomes a smart editing tool.