ClaudeFolio
Lessons

How to Tell When Claude Code Is Hallucinating

Edward Kwun··5 min read
How to Tell When Claude Code Is Hallucinating

Key points

  • Claude Code hallucinations arrive inside work that's 95 percent correct, in the same confident voice it uses when it's right, so the skill is detection, not prevention.
  • The most common hallucination is fake verification: outcome claims with no command output, test results, or evidence attached, and the counter is no receipts, no belief.
  • Invented APIs and version drift are the classic code fabrications, and suspicion should scale with obscurity, with anything version-specific checked against current docs.

When people hear "AI hallucination" they picture the AI citing court cases that don't exist, invented historical facts, confident nonsense about wine pairings. Claude Code hallucinations are a different animal. They're quieter, they're wearing a hard hat, and they arrive inside work that's 95 percent correct, which is exactly what makes them dangerous. The model doesn't say anything obviously insane. It says something plausible, in the same confident voice it uses when it's right, and the whole game is noticing which one you just got.

You can't prevent hallucinations. They're an operating cost of the technology, like turbulence. What you can do is get good at smelling them early, and after a lot of hours in these sessions I can tell you the smells are surprisingly consistent. Here are the five tells I actually watch for, and the habits that catch what the tells miss.

Tell one: claims without receipts

The most common Claude Code hallucination isn't fake code, it's fake verification. The agent says "All tests pass" when it never ran the tests. It says "Done, the feature works end to end" when what actually happened is it finished writing code that looks like it should work. Watch for outcome claims with no evidence attached: no command output, no test results, no screenshot, just a satisfied summary.

The counter is a house rule of mine: no receipts, no belief. When it claims something works, say "run it and show me the output." A model that actually ran the tests pastes the results without friction. And the beautiful thing about an agent, as opposed to a chat window, is that it CAN produce receipts, it has a terminal. Make it use it. If demanding evidence feels rude, remember you're not accusing anyone of lying, you're just being the kind of engineer who reads the gauge instead of trusting the paint job.

How to Tell When Claude Code Is Hallucinating

Tell two: the API that's too good to be true

The classic code hallucination is the invented interface: a method that doesn't exist, a config option that sounds exactly right, a library import nobody ever published. These are seductive because they're not random, the model generates what the API should look like if the world were well designed. Sometimes the world is well designed and the guess is right. Sometimes you get "just set retryOnFailure: true in the client options" and no such option has ever existed.

Suspicion should scale with obscurity. Core language features and famous libraries, the model knows cold. The moment you're in a niche SDK, a young framework, or, worst of all, a specific version of anything, you're in hallucination country. Version drift is its own sub-genre: the model's knowledge has a cutoff, libraries keep moving, and it will confidently hand you last year's syntax for this year's release. Anything version-flavored gets checked against the actual docs, and the lazy way to do that is to make the agent do it: "before you use that option, check the current docs for this version." It can browse. Make it.

Tell three: the summary doesn't match the diff

This one separates people who catch hallucinations from people who ship them. The agent narrates its work as it goes, and the narration is a story about the diff, not the diff itself. Sometimes the story is aspirational: it says it added error handling and the diff shows a renamed variable. It says it fixed the bug in the payment path and the diff touches a README. The words and the changes have drifted apart, and the words are the hallucination.

The fix costs thirty seconds: read the diff before you approve it, every time, and check that what changed matches what it says changed. We've written before about why you have to understand what you shipped, and this is the transactional version of that rule: never let the narration stand in for the change.

Tell four: the instant fold

When you doubt an answer, push on it: "are you sure that's right?" A model on solid ground holds its position and explains why. A hallucinating model folds instantly and completely, apologizes, and produces a different answer with the exact same confidence as the first one. That frictionless reversal is the tell. Real knowledge has some spine; confabulation is a people-pleaser.

Related: if you ask "how do you know that?" and the answer is a restatement rather than a source, a file it read, a command it ran, a doc it fetched, treat the claim as decorative until proven otherwise.

Tell five: the death spiral

Hallucinations compound. The made-up method fails, the model invents a reason the failure happened, the fix for the invented reason breaks something real, and now you're three fictions deep in a session where the model has fully lost the plot. The smell is circularity: the same file getting rewritten for the third time, fixes that undo previous fixes, explanations that keep changing shape. At that point stop prompting. You're not debugging code anymore, you're debugging a story. Roll back to your last good state, clear the context, and re-approach the problem fresh with a tighter description. Ten minutes of restart beats an hour of arguing with a fiction, and it's the reason commits are your save points.

The habits that catch the rest

The tells catch most of it. For the remainder, three habits. First, calibrate your verification to the blast radius: boilerplate and UI tweaks can ride with light review, while anything touching auth, payments, or data deletion gets the full receipts treatment, since that's exactly the territory where a confident hallucination becomes a breach. Second, for anything high-stakes, get a second opinion from a different model, because two models rarely hallucinate the same way, and disagreement between them is a flare marking exactly where you should look. Third, end significant sessions with a verification lap: "list everything you claimed to have done this session, and verify each one with a command." It's a beautiful little trap. Real work survives the audit. Hallucinated work gets caught by the model itself, which is always a slightly surreal thing to watch.

Calibrated distrust is the whole skill

None of this is a reason not to build with Claude Code. I hand it real work every day and it's right far more than it's wrong. The point is that "usually right, occasionally fabricating, always confident" is a specific kind of colleague, and working well with that colleague is a learnable skill: believe the code, doubt the narration, demand receipts, watch the diff, and restart when the story stops making sense.

The one mental shift that matters most: the model never says "I don't know." It structurally can't. Which means uncertainty never announces itself, it just wears confidence like everything else does, and detecting it is your job, not the model's. Do that job and hallucinations get demoted from scary failure mode to routine line item. Skip it, and you'll learn the tells anyway, just more expensively.

FAQ

How can you tell when Claude Code is hallucinating?
Common signs include unsupported claims that work is complete, invented APIs, summaries that do not match the diff, instant reversals when challenged, and repeated fixes that create a circular debugging spiral.
Does Claude Code hallucinate?
Yes, Claude Code can confidently invent methods, misstate what it changed, or claim tests passed without actually verifying the result.
How should you verify Claude Code’s work?
Why does Claude Code invent APIs or configuration options?

Related posts

Comments