Godel's shadow: what mathematics says computation can never do
Roger Penrose builds on Godel and Turing to argue that understanding goes beyond algorithms, Edward Frenkel sees paradox and limits at the heart of mathematics, and Stephen Wolfram describes computational irreducibility. Each identifies a different hard limit on what calculation can predict or prove.
Mathematics sets three different limits on calculation, and they are not the same limit. One is about proof, one about what a machine can decide, and one about prediction. All three speakers treat the limits as proven facts and as good news, because each leaves room for something. They disagree sharply on what the limits say about minds.
Roger Penrose and Edward Frenkel both explain Gödel's theorem. Any consistent set of rules rich enough for arithmetic leaves some true statements unprovable. Penrose met it as a graduate student and feared it meant there were things mathematics could never prove. He came to see it differently. A proof is something whose steps a computer can check. You trust the rules only because you understand that they give nothing but truths. Gödel then shows how to build a statement that this trust lets you see is true, though the rules cannot prove it. So understanding why the rules work reaches further than the rules do. Penrose draws the boldest conclusion, that understanding is something no rule system captures: "It's not a computational procedure". He adds that the cerebellum does enormous computation and appears entirely unconscious. He looks for the missing ingredient in physics we do not yet have, and admits there is no theory. The argument is disputed.
Frenkel tells the history. By 1900 many believed all of mathematics could be derived mechanically from the right axioms. Gödel ended that hope in 1931, and Alan Turing proved a similar limit on what a machine can decide. Frenkel finds the result life-affirming, since it leaves room for discovery. He is wary of the claim that everything is computation, but does not say Gödel proves the mind is not, and allows that future notions of computation may change the picture.
Stephen Wolfram's limit concerns prediction. Very simple programs can produce enormously complicated behavior. Even when you know the rules exactly, often the only way to learn what happens is to run every step. He calls this computational irreducibility. It is not total. Science is the story of finding "pockets of reducibility" where we can jump ahead. He applies the idea to computer security, where no sandbox is perfect, and to AI, where building a machine does not mean understanding or controlling it.
On minds they split. Penrose says understanding goes beyond computation. Wolfram thinks minds are computation, and that we are limited observers bound by the same rules. They also differ on whether neural networks show any understanding. These conversations are dated, so check current work before relying on claims about AI.
The limits do not make computation weak, do not stop mathematics, and do not make everything unpredictable.
What to do with it: keep the three limits apart when you hear them invoked. Before trusting a rule system, ask why you believe its rules give only truths. Treat starting assumptions as choices. When a system cannot be shortcut, stop looking for a formula and run it. Look for the pockets where prediction works. Ask what a model was built to care about. Do not assume that building a machine means controlling it. Practice standing back and asking what you are doing. Hold open questions without panic.