The AI passed my first fake ServiceNow workshop with flying colours. That's exactly how I knew the test was rubbish.
Last time I told you why I wrote a workshop from scratch for an application that doesn't exist. This time, the people; because the people were the problem.
My first attempt was complete and utter rubbish, but usefully rubbish.
Every fake participant spoke perfect English without filler words or hesitation, took their turn gracefully without interrupting, spoke in tidy paragraphs, used the same terminology, had complete confidence, was completely consistent.
Now I've had some fantastic customers over the years but these v1 robo-customers were just frightening. If I ever came across a real customer like this, I'd probably run. Back to the drawing board.
Real workshops don't sound like that, and I've sat in hundreds of them. I needed a rethink for Version 2.
So picture this; I'm sat on my sofa on a Saturday evening, laptop out, describing pretend people with pretend characteristics and knowledge, before I start holding pretend workshops with them about a real subject.
I'm reading that last paragraph back again now and laughing at the level of bizarre we're reaching. A masterclass in talking to myself!
Each of them was cast from memory. Not real people; real patterns. The business owner who knows her application inside out but not our jargon. The technical lead with strong opinions and a long memory. The quiet admin who'll inherit whatever we build. Every one of them got a role, a backstory, and a knowledge boundary; the things they'd plausibly know and, more importantly, the things they wouldn't. Then I planted the awkward stuff on purpose; two people who half-disagree about the environments, a hedged answer nobody follows up, a name dropped with no role attached. If the AI could survive my imaginary friends, it might just survive a real Tuesday morning.
This time the mess wasn't written in, it was cast in. Each participant got flaws along with their backstory; one who interrupts, one who trails off mid-sentence, one who says the vendor's name differently every time. Set them loose on the same workshop and out came the hesitations, the cross-talk, the half-finished answers. All the mess of a real session, but nobody scripted a line of it.
The experiments ran in both directions, too. Speech to text for the listening side; text to speech to give my fake participants actual voices. A scripted workshop read aloud by synthetic speakers becomes audio you can stream through the live pipeline exactly like a real meeting; a full dress rehearsal with nobody real in the room.
That fake workshop earned its keep almost immediately, because it found the first real wall. That story next time; it involves an assistant that started forty-five seconds behind the room and ended four minutes behind it.
If you've built your own test rigs (or been tempted to test on the real thing), give me a shout in the comments.
First published on LinkedIn, 15 July 2026. The conversation lives there; the writing lives here.