LLMs and hallucinations

The tool behind the chatbot

When you type a question into a chatbot and it answers in fluent sentences, you are talking to a large language model (LLM). It is the technology behind tools like ChatGPT, and it is a kind of AI trained on enormous amounts of text. Knowing roughly how it works explains both why it is so useful and why it sometimes gets things confidently wrong.

It predicts the next word

An LLM does one core thing extremely well: it predicts the next word. Trained on a vast amount of writing, it has learned which words tend to follow which, and it builds an answer one word at a time, each choice guided by everything before it. Do that fast enough, over billions of examples, and the result reads like a thoughtful human reply.

This is worth sitting with, because it is the key to everything else. The model is not looking up facts in a database, and it is not reasoning about the world the way a person does. It is producing the most plausible sequence of words for your question. Usually the most plausible words are also true, because true statements are common in its training text. But plausible and true are not the same thing, and that gap is where the trouble lives.

A hallucination is a confident wrong answer

When an LLM produces something that sounds right but is false, that is called a hallucination. The unsettling part is not that it makes mistakes, all tools do, but that it makes them with total confidence and perfect fluency. There is no nervous tone, no hedging, no visible seam between a solid fact and an invented one.

Hallucinations show up as invented statistics, quotes no one ever said, legal cases that do not exist, a book credited to the wrong author, or a made up phone number. Because the model's job is to sound plausible, a wrong answer is written in exactly the same assured style as a correct one. You cannot tell them apart from tone alone.

Why it happens, in plain terms

Remember what the model is doing: predicting likely words, not checking truth. If your question points towards an answer that does not exist, the model will still generate the most likely sounding words to fill the gap, because that is all it knows how to do. It has no built in sense of "I do not actually know this." Fluency is not knowledge, and confidence is not accuracy.

Always check facts that matter

None of this means an LLM is useless. It is a genuinely powerful assistant for drafting, summarising, explaining and brainstorming. It just means you should treat its output as a confident draft, not a verified source.

The practical rule is simple: the more a fact matters, the more you verify it elsewhere. A name, a number, a date, a medical or legal or financial detail, a quote you plan to repeat, check these against a trusted, independent source before you rely on them or pass them on. Let the model speed up your work, but keep the final judgement about what is true firmly with you.

Check yourself