AI is reliably useful for two parts of user research right now: turning a plain-language question into a structured study, and turning a pile of open-ended responses into a clear summary with the original quotes attached. It is not useful, and shouldn’t be trusted, for two other things: standing in for a real participant, or deciding what a finding means without a person checking the reasoning behind it. Almost everything people argue about when they argue about “AI in research” is a version of one of those four claims.
Where AI genuinely helps: turning a question into a study
Writing a good study is a skill most teams don’t have lying around: choosing the right method for the decision at hand, phrasing tasks so you’re not leading the participant, deciding how many questions is too many before people start clicking through on autopilot. A model that’s seen enough of this structure can take “I want to know if people understand our new pricing page” and turn it into a specific set of tasks and questions built from an established method, ready for a human to review before anything goes out. That’s genuinely useful compression: the model isn’t inventing research methodology, it’s applying patterns that already exist, faster than a person would look them up.
Where AI genuinely helps: reading responses back into findings
The other side of the same coin: once responses come back, someone has to read them, group the recurring complaints, and separate a real pattern from one loud participant. A model is good at exactly this kind of compression across a large pile of open-ended text, and good at keeping the original quote attached to the summary it produced, so a human can check the model’s reasoning against what was actually said rather than trusting a paraphrase on faith.
Where AI does not help: it has no lived experience
A model can describe what confuses people, drawing on patterns from things it’s read. It cannot be confused itself. It has never clicked the wrong button because a label was ambiguous, never felt the specific frustration of a checkout flow that asks for the same information twice. Every insight it produces is a compression of things real people already said or did. It has nothing of its own to add about how a genuinely new interface feels to use, because it has never used one.
Where AI does not help: synthetic respondents aren’t respondents
A growing slice of the research-tools market now sells AI personas trained to answer as if they were real customers, marketed as a faster, cheaper stand-in for actual people. The important distinction is not subtle: a human answer is a direct observation; a synthetic answer is a model’s approximation of what a person might say. AAPOR’s recent guidance puts the point plainly: calling both of them respondents blurs what is being measured. That is a category mistake, not just a question of whether one version is a little less accurate.
The risk is not only that a synthetic panel gets an answer wrong. It can produce an answer that feels reassuringly coherent and agrees with the direction you already hoped to take. In a comparison of human and synthetic survey participants across three countries, synthetic participants tended to give more positive and less negative answers than the human participants they were meant to approximate. That does not make every synthetic answer useless. It does make them a poor substitute when the question is whether a real customer will push back, misunderstand the offer, or simply feel the friction you failed to anticipate.
The training data brings another boundary. AAPOR notes that these systems draw on massive but not necessarily representative training corpora, then on alignment choices made after training. A persona can sound specific without carrying the context, culture, language, constraints, or inconvenient edge case that makes a real person’s answer matter. The smoothness is part of the trap: an answer can read like a customer quote without being evidence from a customer.
There is still an honest place for the tool. Use it before fieldwork to pressure-test a questionnaire: whether the wording makes sense, the logic holds together, and the skip patterns work. That is also where Gallup places the opportunity: refining questions and hypotheses before they go in front of real people, while separately validating where simulated outputs hold up. Pilot with synthetic; validate with humans. Once the decision depends on what people actually think, do, notice, or resist, put the question in front of them.
AI is very good at compressing what people already said. It has never been good at generating what people would say, and there’s no evidence that changed.
The honest way to use AI in a research workflow
The practical line holds up well in practice: let AI handle the drafting and the reading-back, and keep the actual participants real. That split doesn’t require a paid tier to access, either. Fiuto runs on Anthropic’s Claude models on every plan, including Free, and every AI feature there is metered by usage credits rather than gated behind a subscription tier: which model handles a given task depends on the task, not on what you’re paying. What stays constant across every plan is that responses only ever come from people you invited yourself, never a generated persona standing in for them.
A line worth drawing
If a claim in your findings traces back to something a real person actually said or did, AI compressed it faster than you would have by hand, and that’s worth using. If a claim traces back to a model imagining what someone might have said, it isn’t a finding. It’s a guess wearing a finding’s clothes, and the fix is the same one it’s always been: put the question in front of real people, and let the model help you read what comes back.
