That AI Tool Promising to Replace Your User Research? Not Yet.

Hey,

Quick apology—last week I typed “MeKinsey.” That’s the parallel universe version ;)

One update: Starting January 1st, paid subscriptions move to $75/year ($7.50/month). I highly recommend a yearly subscription, locking in the current rate before year-end, which gets you the best deal.

On top of the paid-only content, I offer a 30-minute advisory/ Q&A session for upgraded subscribers. The questions that members typically ask are:

• How do you see AI adoption today?

• How individual and human behaviour, as a group, is influenced by AI, especially in the workplace.

• How much has AI impacted the job market?

• Is the AI bubble going to burst? Which AI model company will outlast the others?

And any other questions you’ve read and would like clarification on.

I was interviewed by AI Ready CMO last week on several AI behaviour fundamentals. They offer daily AI-in-marketing intel and a weekly deep dive, which is powerful if you’re tired of your marketing team bringing you every shiny new tool. Check it out.

Thanks, and enjoy the article.

Jing

---

Someone on your team went to an AI meetup and brought the term “synthetic user panels” back to your weekly catch-up.

Or… maybe you’ve seen a headline about how AI can now predict purchase intent with 90% accuracy. Or… that you have a vendor who pitched to you about using AI to help you set the direction for your business strategy.

Regardless, all you heard is how some of the latest platforms can simulate thousands of customers’ opinions or even mimic their purchasing behavior in hours for a fraction of traditional research costs.

The promise is obvious to C-levels or VPs like you.

Why spend $50K and six weeks when you can get the “near equivalent” insights by Friday (and it’s Thursday night now)?

By the way, this product likely also makes your team happy.

So they no longer have to sit in awkward user interviews, staring at the users, asking prepared questions, or listening to customers complain, or replying with entirely irrelevant answers to their questions… while taking notes and trying to wrap it as a 'useful’ conversation.

At the end of the day, your team still doesn’t have an answer. What they got is only a bunch of chaotic chat with users sitting in their Confluence or some cloud drive that still needs to be analysed and sorted through.

So let’s go with synthetic user data! Why not? Happy days, it saved everyone the trouble in the process.

TLDR

• Q: If I say no to synthetic users, will I look like a dinosaur soon?
  No (if you ask me), you’ll look like someone who thinks things through.
  The research shows 90% reliability (sounds more than good enough) for mainstream products with well-documented customer patterns. However! For anything novel, strategic, or differentiated, you’re in the high-error zone.
  I have other questions if novelty is not part of your strategy (smirk).

• Q: Who’s accountable when decisions based on synthetic data turn out wrong?
  You. There’s no doubt about this.
  The vendors disclaim accuracy.
  The researchers explicitly warn that these are “not plug-and-play replacements.” If your team builds a strategy on synthetic insights and it backfires, the tool won’t be blamed—your judgment will.

• Q: Are there any scenarios where synthetic users can be useful?
  Yes, when testing theories, hypotheses, or screening ideas (similar concepts).
  Narrowing 100 ideas, like ads or product concepts, to 10 before real testing. Surface-level pattern matching in mature, slow-moving categories. Anything exploratory, not conclusive.

• Q: Is it still cheaper to use synthetic users even after considering the risks? 
  Depends on what you’re optimizing for.
  Cheaper per insight? Absolutely. Cheaper per successful product? Only if you’re in a mature, slow-moving market where yesterday’s users predict tomorrow’s.

Many of you are business owners with skin in the game. Your employees might stick around for a couple of years. You’re betting everything.

If you have only one more second to spare, then this:

Do NOT use or even try synthetic users in 2025-2026.

When no one else on your team bothers to ask the difficult questions, you should.

For instance, while the synthetic user achieves 90% of human test–retest reliability, what about the other 10%?

Shall we?

Here’s a YouTube version (released on Wednesday). Still, only paid subscribers have the full reveal of the insight!