Why LLMs make things up, and three ways builders fix it
Stephen Wolfram, Yann LeCun, Marc Andreessen, Aravind Srinivas and Jimmy Wales explain hallucination as a consequence of how language models generate plausible text, and point to computation, retrieval with citations and human-checked sources as remedies. Knowing the cause helps users judge when an answer can be trusted.
Language models make things up because of what they are built to do: continue text in a way that is typical of what people have written, one token at a time, with no separate step that checks the result against the world. Builders respond in three ways, and each one works by moving the factual claim out of the generator. The practical rule is to trust an answer as far as you can see where it came from.
Stephen Wolfram, the founder of Wolfram Research, describes the mechanism as wide and shallow, while real computation is deep. The model writes fiction and fact with the same machinery, so his advice is blunt: "the idea that you're going to get factual output is not a very good idea." Yann LeCun, Meta's Chief AI Scientist, adds the arithmetic. Each token has some chance of going wrong, so the probability of staying correct shrinks as an answer gets longer, and the space of unusual prompts has a very long tail. We believe the output anyway, he says, because "we're fooled by their fluency." Jimmy Wales, the Wikipedia co-founder, says a model does not really know or care about the truth.
The first fix is Wolfram's: hand exact questions to a computational system. The model translates a request into precise code, a deterministic system produces the answer, and a person reads the code. The second is retrieval with citations, which Aravind Srinivas built Perplexity on. The system searches, extracts relevant paragraphs and tells the model to answer only from them, with footnotes. The third is anchoring answers in sources people have checked, as Wikipedia does with its verifiability rules.
The experts agree on the cause far more than on the cure. LeCun thinks the flaw belongs to the architecture and wants systems that plan an answer before turning it into text. Srinivas bets that better models, fresher indexes and better retrieval will keep driving errors down. Marc Andreessen points out that the same behavior is called creativity when we like it, and that a law firm told his team it wants creative models and then checks the citations separately. He also warns that anchoring a model in approved sources makes it as right, and as wrong, as the people doing the approving. Sam Altman notes that tuning with human feedback makes a model more usable, which is different from making it check facts. The article adds that Wolfram and Srinivas are describing their own products and that LeCun is an investor in Perplexity.
No fix removes the need to look. Model-written code must still be read, a citation can point to a stale or irrelevant page, and a cited source can itself be wrong. Where an error would be costly, as in legal, medical or financial matters, check the primary source or ask a qualified professional.
What to do with it: treat a bare chatbot answer as a fluent draft, not a fact. Send exact questions to a tool that computes. Prefer answers with footnotes, and open the sources for any claim you plan to rely on. Be more suspicious of long answers and unusual questions. Show the model its whole answer and ask it to check. Separate creating from checking, and do not let fluency stand in for accuracy.