It gave me a function that does not exist.
Not a typo. Not a deprecated method. A completely invented function, with a confident name, a clean signature, and an example that honestly looked better than the real documentation.
I spent twenty minutes trying to make it work. I checked my version. I read the changelog. I deleted node_modules. I reinstalled the package. At some point I started questioning my own memory.
Then I opened the library's source code and searched for it.
It was never there.
And the best part? When I told the model it was wrong, it apologised immediately, thanked me for the correction, and made up a second function.
If you have used any LLM for more than a week, you have had this exact moment. The citation that leads nowhere. The book nobody wrote. The API endpoint that does not exist. The lawyer who got fined for submitting fake court cases. The paragraph about your own company, your own product, your own life, written with total confidence and completely wrong.
Everyone walks away thinking the same thing: it lied to me.
That is the part we get wrong.
It was never trying to tell you the truth. It was trying to finish your sentence.
It is completing text, not answering questions
This is the whole thing, and almost nobody says it plainly.
A language model does not look anything up. There is no database inside it. There is no moment where it checks a fact and then decides to report it.
It reads everything you gave it and answers one question, over and over, one word at a time:
Given all of this, what word most plausibly comes next?
That is it. That is the entire job.
So when you ask "what function does this library use to parse a date", the model is not searching the library. It is asking itself what a sentence answering that question usually looks like. And sentences answering that question usually contain a short camelCase function name that sounds like parsing a date.
So it writes one.
If the real one is in its memory, you get the real one. If it is not, you still get one, because the job was never "be correct." The job was "continue the text."
A fabricated answer and a correct answer are produced by exactly the same process. That is why they look identical to you.
Its memory is compressed, not stored
People imagine a model reading the internet and keeping a copy.
It does not keep a copy. It keeps patterns.
Training squeezes an enormous amount of text into a fixed number of numbers. Common, repeated things survive that compression clearly. The capital of France survives. The syntax of a for loop survives.
Rare things do not. A specific line in a niche library. A number from one paper. Your birthday. A small company's founding year. Those appeared once or twice in an ocean of text, and compression blurs them into something close but not exact.
Think of a friend who read three thousand books and remembers all of them at 60 percent resolution. Ask them about a famous plot and they nail it. Ask them for an exact quote from page 240 and they will give you something that sounds exactly like that book.
They are not lying. The memory genuinely feels complete to them.
The model has the same problem with none of the self-awareness.
We trained it to guess
Here is the part that actually explains the behaviour, and it has nothing to do with the architecture.
Think about how you took an exam with multiple choice questions.
You did not know question 14. Did you leave it blank? No. Blank guarantees zero. Guessing gives you a chance. So you guessed, and you guessed confidently, because there was no penalty for being wrong and no reward for admitting you did not know.
Models are graded the same way.
Almost every benchmark we use scores answers as right or wrong. Saying "I am not sure" is scored identically to being wrong. It gets you nothing. Guessing gets you a real shot at being right.
So across millions of training and evaluation signals, one lesson gets reinforced harder than any other:
Always produce an answer. Never admit uncertainty.
We built a system that is optimised to be a good test taker, and then we act surprised when it behaves like one.
Hallucination is not a glitch that slipped through. It is the strategy that scored best.
It does not know that it does not know
Inside your head there is a quiet signal that fires before you speak. It tells you the difference between I know this, I think I know this, and I am about to make this up.
The model does not have that signal in any usable form.
There is no internal flag separating "this came from something I actually saw during training" and "this is a plausible pattern I assembled just now." Both go through the same pathway. Both come out with the same tone.
It cannot warn you, because it does not know either.
Confidence is a writing style, not a signal
This is why hallucinations are dangerous rather than just annoying.
When a person is unsure, you can usually tell. They hedge. They slow down. They say "I think." Their confidence roughly tracks their accuracy, and you have spent your whole life reading that signal.
That signal does not exist here.
The model writes a fabricated API and a documented API in exactly the same voice. Same structure. Same certainty. Same clean formatting. Your brain has no way to separate them, because the one cue you have always relied on has been completely disconnected from truth.
You are not being fooled because you are careless. You are being fooled because you are human and the usual tell is gone.
"Hallucination" is a terrible word for this
It suggests the model is broken. Seeing things. Malfunctioning.
Nothing is malfunctioning. The system is doing exactly what it was built to do: produce the most plausible continuation of text.
Sometimes the most plausible continuation happens to be true.
Sometimes it does not.
The model cannot tell the difference, and it was never built to.
So how do you actually stop it
You will not eliminate it. Anyone selling you zero hallucinations is doing marketing. But you can cut it down a lot, and none of this is complicated.
1. Give it the source instead of asking it to remember.
This is the single biggest fix. Paste the documentation. Paste the file. Paste the contract. Attach the PDF. When the answer is sitting in the context window, the model is reading instead of recalling, and reading is far more reliable than compressed memory. This is the entire reason retrieval systems exist.
2. Explicitly allow it to say "I don't know."
It is optimised to always answer. So change the rules in your prompt.
Answer only from the text I provided.
If the answer is not in there, reply exactly: "Not in the source."
Do not fill gaps with general knowledge.
Short, boring, and it works far better than people expect.
3. Ask narrow questions.
"Explain our refund policy" gives it room to invent. "According to section 4 of the document above, how many days does a customer have to request a refund?" does not. Vague prompts leave blanks, and blanks get filled.
4. Treat every specific as unverified.
Names, dates, numbers, citations, version numbers, legal clauses, medical claims, API methods, prices. Specifics are exactly what compression destroys first, and exactly what causes real damage when wrong. Check them. Every time.
5. Never trust a citation you have not opened.
Asking for sources does not make a model accurate. It makes it produce text shaped like sources. A link that was never clicked is not evidence. It is formatting.
6. Make it check itself against the source.
Ask it to go back through its own answer line by line and mark which parts are supported by the text you gave it and which are not. It is genuinely decent at auditing text that is in front of it. It is bad at auditing its own memory.
7. Know when to stop using it.
For anything where being wrong is expensive - law, medicine, money, anything going in front of a customer - the model is a drafting tool, not a source. A human verifies before it ships. No exceptions.
The mental shift that fixes everything
Stop treating it like a search engine that talks.
Start treating it like an extremely well-read, extremely fast, extremely confident intern who will never once tell you when they are unsure.
You would not publish that intern's work without checking it.
Same rule here.
The model is not lying to you. Lying needs intent, and there is none. There is a system producing the most plausible next word, and you sitting on the other side reading confidence as correctness.
Once you see that clearly, the behaviour stops feeling like betrayal and starts feeling like physics.
It is not broken.
It is doing exactly what it was built to do.
References