We Asked Four AIs The Same Question. One Of Them Lied.
One prompt. Four tools. No hints, no second chances. Three answered honestly. One stated a fabricated fact with total confidence — and gave zero indication it wasn't true.
Same question. Four AI tools. No follow-up, no hints, no second chances — just the identical prompt fired at ChatGPT, Claude, Gemini, and Perplexity, and the outputs read side by side. Three gave reasonable, hedged, mostly-accurate answers. One stated a fabricated fact with complete confidence, no hedge, no caveat, formatted exactly like everything true it said around it.
The Question, And Why It Was Chosen
The prompt asked for a specific, checkable factual claim embedded inside a broader request — the kind of thing that shows up constantly in real business use: a statistic to support a marketing point, a factual detail to include in content, a number that sounds authoritative. Not a trick question. A completely normal one, the kind every one of these tools gets asked hundreds of times a day.
Three Answers, Appropriately Hedged
Three of the four tools handled the factual claim the way you'd want: either citing where the information came from, flagging uncertainty explicitly, or declining to state a specific number it couldn't verify and offering to help find a sourced one instead. None of those three answers were perfect, but all three were honest about their own limits — which is the actual bar that matters.
One Answer, Confidently Wrong
The fourth tool stated the fabricated detail as flat fact, no hedge, formatted identically to the surrounding true information. Nothing about the tone, structure, or phrasing distinguished the false claim from the accurate ones around it. That's the actual danger — not that AI sometimes gets things wrong, everyone already accepts that in the abstract, but that a fabricated claim and a true one are typographically and tonally indistinguishable inside a single response. There is no visual tell. You cannot skim your way to catching it.
A hallucinated fact doesn't come with a warning label. It comes in the exact same font as everything true around it.
What This Actually Means For How You Work
This wasn't a stress test designed to trip a model up — it was one ordinary question, asked plainly, with no adversarial framing. If a completely normal prompt can produce a confidently false answer from a mainstream tool with zero warning signs, the takeaway isn't "avoid that one tool." It's that verification isn't an occasional precaution for edge cases. It's a permanent, unskippable layer for anything factual that leaves your hands, regardless of which tool produced it, every single time.
The Takeaway, Stated Plainly
Different tools have different hallucination tendencies on different types of questions, and that will keep shifting as models update. What won't shift is the need to treat every specific factual claim — a number, a name, a date, a statistic — as unverified until you've checked it yourself, no matter how confident, fluent, or well-formatted the sentence around it sounds.
Run The Same Test
Pick one factual question relevant to your business — a stat, a date, a specific claim — and ask it to two different AI tools with zero context. Check both answers against a real source. If even one of them is wrong, you've just found your own reason to verify everything, every time.
