When not to use it
This chapter closes the course, and it is the most useful one. A model produces a plausible answer, never a checked answer. All discernment therefore comes down to one question, asked before you ask anything of it: what happens if the answer is wrong and nobody notices?
The previous chapters have shown a mechanism: a model cuts text up, works out a distribution, draws from it, and starts again. Nothing in that mechanism checks anything at all. An answer comes out because it is probable, not because it is right, and the model has no reliable signal of its own ignorance. This chapter draws the practical conclusion.
The right question is never “can I?”. It is “who pays if this is wrong?”.
The three tests
Three questions are enough, and they are asked in this order. The first one that fails closes the door.
The cost of an error
Can the error be put right? A badly worded draft is rewritten in two minutes. A medicine, a tax deadline, a diagnosis, a dosage, a message sent to a client are put right badly or not at all. The more irreversible an error is, the less place a model has in the decision.
Whether it can be checked
Can you check the answer at its source, yourself, in a reasonable amount of time? If so, the model is an acceptable starting point, and the check remains compulsory. If not, the answer is unusable, however confident it sounds. An answer that cannot be checked is not half an answer: it is no answer at all.
Beware of checks that only look like checks. Asking the model whether it is sure checks nothing: the confirmation is worked out by the same mechanism as the answer, and is just as plausible as it. Asking for its sources checks nothing either, until those sources have been opened one by one, because an invented reference is perfectly well formed.
Legitimacy
Does the situation call for a human being who answers for it? A decision about a person, about their rights, their work or their marks commits someone who must be able to answer for it and who can be challenged over it. No machine carries that responsibility, and delegating the decision to one does not make the responsibility disappear: it only makes it impossible to find.
The cases where the tool becomes a trap
- Deciding on a treatment, a dosage or a diagnosis, outside a health professional.
- Basing a legal act, a contract, a declaration or an appeal on an answer not checked against the official source.
- Settling a decision about a person: recruitment, marking, discipline, the award of a benefit.
- Accusing someone, on the basis of a detector or an impression.
- Processing content you have no right to hand over: see the chapter what you give it.
- Replacing an exact calculation with a prediction: a language model does not count, it predicts what a count looks like.
- Making a first draft you are going to rewrite anyway.
- Rewording a text you wrote yourself and will read again.
- Getting leads to check, knowing that none of them is settled.
- Explaining an idea you are able to cross-check elsewhere.
- Translating in order to understand, never to publish without a reading by someone who speaks the language.
The detector trap
One case deserves treatment of its own, because it does real damage and presents itself as the solution: the tools that claim to say whether a text was written by an artificial intelligence. This site offers none, and that is not a gap.
Three published findings justify it. An independent evaluation of fourteen detection tools, including commercial tools sold to education, concludes that none is both reliable and accurate. Another study establishes that several widely used detectors wrongly classify the majority of texts written by non-native speakers of English as machine-generated, while handling the texts of native speakers correctly: the error is not spread at random, it falls on people who are already at a disadvantage. Finally, the company that had published the best-known detector withdrew it six months after launch, itself citing a rate of accuracy too low to be useful.
A detector’s result is not proof and grounds no penalty. A doubt about a piece of work is dealt with by a conversation, by a question about the method followed, by an oral exchange, by asking for an intermediate draft. Never by a percentage.
The detail of this evidence, and what a school can put in its place, are covered on the blog: AI detectors at school.
The material cost, which counts too
One last reason to hold back has nothing to do with accuracy. Using a general-purpose generative system for a task that a specialised tool does better costs, for that same task, an order of magnitude more energy and emissions. Searching for an exact string in a document, sorting a list, adding up a column, converting a format: those tasks have tools that do them exactly, faster and without invention. Worldwide, the electricity consumption of data centres tied to artificial intelligence is growing around four times faster than total electricity demand, and is expected to more than double between 2024 and 2030.
The move here is simple: ask first whether the task has an exact answer. If it has one, an exact tool exists.
The whole course in one rule
A model is for producing what you are going to check, never for deciding what you will not check.
The rest follows. If checking is impossible, the use does not happen. If checking is possible and you do not do it, the model is no longer what is at fault.
Going further
The mechanism of invention is covered in the chapter why it makes things up. The checking routine, step by step, is published on the blog: the checking routine. Obligations specific to a region enter as a module beside the course, never as the core: see the European obligation and the official frameworks. The contents of the course are on the understand page.
Sources
- Testing of detection tools for AI-generated text Debora Weber-Wulff, Alla Anohina-Naumeca, Sonja Bjelobaba, Tomas Foltynek, Jean Guerrero-Dib, Olumide Popoola, Petr Sigut, Lorna Waddington, 2023. Establishes that an independent evaluation of fourteen tools for detecting AI-generated text, including commercial tools used in education, concludes that none of them is both reliable and accurate, with a bias towards classifying text as human rather than detecting the machine.
- GPT detectors are biased against non-native English writers Weixin Liang, Mert Yuksekgonul, Yining Mao, Eric Wu, James Zou, 2023. Establishes that several widely used detectors wrongly classify the majority of texts written by non-native speakers of English as AI-generated, while identifying the texts of native speakers correctly.
- OpenAI scuttles AI-written text detector over “low rate of accuracy” Devin Coldewey / TechCrunch, citing OpenAI’s official addendum of 20 July 2023. Establishes that OpenAI withdrew its own tool for detecting AI-generated text six months after launch, itself citing a rate of accuracy too low to be useful.
- Energy and AI International Energy Agency, 2025. Establishes that worldwide electricity consumption of data centres tied to AI is growing around four times faster than total worldwide electricity demand, and is expected to more than double between 2024 and 2030.
- Power Hungry Processing: Watts Driving the Cost of AI Deployment? Alexandra Sasha Luccioni, Yacine Jernite, Emma Strubell, 2024. Establishes that general-purpose generative AI systems consume, for a given task, an order of magnitude more energy and carbon emissions than a system specialised for that same task.
- Why Language Models Hallucinate Adam Tauman Kalai, Ofir Nachum, Santosh S. Vempala, Edwin Zhang, 2025. Establishes that current training procedures, and above all current evaluation procedures, reward guessing a plausible answer over admitting uncertainty.