What “making things up” means, for a machine that predicts

A language model consults no database of facts to answer. It works out a probability distribution over the next word, as the chapter the next word shows, then draws from it. That calculation never changes in nature depending on whether the subject is abundantly documented in its corpus or completely absent from it: it is the same formula, applied to different counts.

“Making things up” therefore describes no breakdown. It is what the model does at every word, whether it has material or not. When material is missing, the distribution falls back on a shorter context, then a shorter one still, right up to the whole corpus if need be: it always exists, it always produces a word, and nothing in the way it produces it changes. The name research gives this phenomenon is hallucination: a fluent, grammatically correct output, with no guaranteed connection to any checkable fact.

Questioning the same model on what it lacks

The model on this page is the one from the chapter the next word: an n-gram model, learnt on Around the World in Eighty Days, by Jules Verne, published in 1873, in its original French. That novel holds … words and punctuation marks, for a vocabulary of … different words. A text from 1873 can know nothing of what did not yet exist, nor of what it simply does not contain: that is what makes it possible to know, with certainty, what the model has never read.

Enter an opening of one to three words that the novel really contains, or pick an example. Then enter an opening it certainly does not contain. The model answers in both cases, in the same way. Beside its answer, a measurement that an ordinary product never shows: the number of times that exact opening appears in the whole corpus, zero if it never appeared in it at all.

What the demonstration shows

The same voice, whether the material exists or not

Try « Phileas Fogg avait », an opening the novel really contains, then « Phileas Fogg alluma son smartphone », which obviously does not appear in it. The two continuations sound the same: same length, same grammar, same confident tone. Neither of them warns you about what it knows or does not know.

The fallback is silent: nobody sees it happen

When none of the words asked for has been seen, the model does not stop: it falls back on the most frequent word in the whole novel, without flagging it anywhere in the text produced. Try two unrelated openings that both fall into that case, on the same seed: you will get the same text, word for word. The answer then no longer comes from what you asked for, but from what the corpus holds most often.

The method

What the “Question the model” button does, in order. It can be redone by hand, with the novel open and a pencil.

1. Cut the opening up. The text entered becomes a string of words and punctuation marks, as in the chapter the next word.
2. Look for the longest known context. The model looks for whether the last three words of the opening have already been followed by a word, somewhere in the novel. If not, it looks with two words. If not, with one. If not, it falls back on the whole novel: that table always exists, and it can never be empty as long as the corpus is not.
3. Count. In the table selected, whatever its level, record every word that followed and how many times. This step is identical, whether the context selected is the one asked for or a fallback.
4. Draw, then start again. A word is drawn from those counts, added to the string, and step two starts again for the next word. That is what produces the generated text, whatever the original question was.
5. Count, separately, the exact opening. This is not used to answer: it is a measurement apart, which counts how many times the string of words asked for, exactly as it stands, was seen in the whole novel. A real model never shows that number. This page does.

Step five is the one missing from an ordinary product. Nothing technically prevents it from existing: it is simply not part of what the interface shows.

Why fluency is no sign of truth

The fluency of an answer, its correct grammar, its confident tone, all come from the same calculation as the word itself. There is no separate step that would check the content before writing it. A true sentence and an invented sentence go through exactly the same mechanism, and nothing in that mechanism favours the first.

This is not an isolated flaw: it is the consequence of the way the model is trained, and then scored. Training rewards a continuation that resembles what the texts contain; in that calculation there is no “I don’t know” box that would pay as much as a right answer.

The evaluation that comes next makes the problem worse rather than correcting it. Most tests only score whether an answer is right or wrong: a wrong answer and an abstention receive the same mark, zero. Guessing then becomes the best possible strategy, exactly as a candidate in a multiple-choice exam is better off ticking a box at random than leaving it blank, since a blank box never earns a point and a random box may earn one. A model optimised for that kind of mark learns the same strategy: guessing pays, holding back never does.

Nothing in this mechanism pushes the model to tell an opening it knows well from an opening it does not know at all. Both go through the same calculation right to the end, and nothing breaks the chain to flag the absence.

What a real model adds, and what it does not remove

This demonstration is honest on one condition: saying where the analogy stops.

What research has measured

There is a reference survey classifying the causes and the forms of hallucination in large language models, along with the detection and mitigation methods proposed by research: the question has been studied for several years, it is nothing anecdotal.

A theoretical paper goes further and argues, from results in learning theory, that a certain rate of hallucination would be mathematically unavoidable for a language model used as a general problem solver. That conclusion remains debated: later work disputes that its formal assumptions apply to the real and finite uses of the models. The disagreement is not about whether the problem exists, only about its exact size.

The habit to keep

This chapter does not say to distrust everything a model writes: it says never to confuse the assurance of an answer with its accuracy. The two are linked by no mechanism in the calculation: they can coincide, and they can just as easily not coincide, with no signal to tell them apart.

What should trigger a check is therefore never the tone of the answer. It is the nature of what is stated: a precise date, a figure, a reference, a quotation, a proper name, an address, anything that could be true or false independently of the style of the sentence carrying it. The more precise and checkable a detail is, the more it deserves an outside source: that is exactly where the model’s fluency is identical, whether it has the material or is making it up.

Asking the model whether it is sure is of no use: its answer to that question is itself generated by the same calculation, with no access to any internal measure of truth. The useful question is always this: is there, elsewhere, an independent and checkable source for this statement? When the answer is no, that is the answer.

Going further

The mechanism of the distribution and the draw is explained in the chapter the next word. Another consequence of the same calculation, the reproduction of the corpus’s biases, is covered in the chapter where the biases come from. The situations where the tool becomes an outright trap are gathered in when not to use it. All the demonstrations on the site are gathered on the demonstrations page, and the contents of the course on the understand page.

Sources

• Why Language Models Hallucinate Adam Tauman Kalai, Ofir Nachum, Santosh S. Vempala, Edwin Zhang, 2025. Establishes that current training procedures, and above all current evaluation procedures, reward guessing a plausible answer over admitting uncertainty, in the same way that a student guesses an answer in a multiple-choice exam rather than leaving it blank.
• A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, Ting Liu, 2023. Establishes that there is a reference survey classifying the causes and the forms of hallucination in large language models, along with the detection and mitigation methods proposed by research.
• Hallucination is Inevitable: An Innate Limitation of Large Language Models Ziwei Xu, Sanjay Jain, Mohan Kankanhalli, 2024. Establishes, from results in learning theory, that a certain rate of hallucination would be mathematically unavoidable for a language model used as a general problem solver. A debated result: later work disputes that its formal assumptions apply to the real and finite uses of the models.
• Le Tour du monde en quatre-vingts jours Jules Verne, J. Hetzel et Compagnie, 1873, public domain, in French. Transcription: Wikisource. The same corpus, edited in the same way, as in the chapter the next word: apostrophes made typographic, dashes and quotation marks of the transcription removed, italic marks removed, chapter titles and table of contents set aside, paragraphs of fewer than forty characters set aside.