Why LLMs make things up, and three ways builders fix it
October 5, 2026 · 14 min read
Stephen Wolfram, Yann LeCun, Marc Andreessen, Aravind Srinivas and Jimmy Wales explain hallucination as a consequence of how language models generate plausible text, and point to computation, retrieval with citations and human-checked sources as remedies. Knowing the cause helps users judge when an answer can be trusted.
The rules, numbers and steps worth keeping in front of you.
- Models generate likely text one token at a time; nothing in that process consults reality.
- Wolfram: treat a chatbot as a linguistic user interface, not a source of facts.
- LeCun: per-token errors compound, so longer answers are more likely to go wrong.
- LeCun: unusual prompts sit in a very long tail that fine-tuning does not cover.
- Fix one (Wolfram): send exact questions to a computational system, and read the code the model wrote.
- Fix two (Srinivas): retrieve documents and make the model cite everything; it should say nothing it did not retrieve.
- Fix three (Wales): answer from sources people have checked under verifiability rules.
- Srinivas's four failure points: the query was misread, snippets are stale or thin, irrelevant detail confused the model, or the documents are off topic.
- Andreessen: let the generator create, then verify separately, as the law firm does with citations.
- Wales's warning: a checked source can still be wrong; a vandalized line about him became a published claim.
- Wolfram: show the model its whole answer and it can spot errors it made along the way.
- Altman: the output that gets attention is the rare worst one, so try the question yourself.
- For legal, medical or financial matters, check the primary source or ask a qualified professional.
This is the short form. The full insight has every expert's reasoning, their own words and the sources.