Nick Bostrom on the Simulation Argument, Anthropic Reasoning, and Superintelligence

High-Level Overview

Lex Fridman talks with Nick Bostrom, the Oxford philosopher who directs the Future of Humanity Institute and wrote Superintelligence. Most of the conversation, about an hour and forty minutes of it, is a careful walk through Bostrom's simulation argument: what it actually claims, what it assumes about technology and consciousness, and how the probabilities shift between its three possibilities. The last part turns to superintelligent AI, where Bostrom is keen to talk about the upside as well as the risks. The conversation was recorded before the coronavirus pandemic.

The tone is patient and precise. Bostrom keeps separating ideas that people tend to run together, and Lex, by his own description, sometimes says dumb things on purpose to push back. Lex is more willing than Bostrom to blur the line between real and simulated, and between seeming conscious and being conscious, and those gentle disagreements give the episode its energy. The flow is methodical: definitions first, then thought experiments, then probability, then AI and the meaning of life.

The Discussion Explores:

• The difference between the simulation hypothesis and the simulation argument
• What a "technologically mature" civilization could do, and why timescales do not matter to the argument
• The three alternatives: near-universal extinction, near-universal loss of interest, or we are probably simulated
• Consciousness in computers, and whether it can be faked
• Nozick's experience machine and what it reveals about what we value
• The doomsday argument and the tricky field of anthropic reasoning
• Why being Elon Musk might be a reason to wonder, and what could lie outside a simulation
• Superintelligence: the intelligence explosion, the neglected upside, and what a good future could look like

Deep Dive into Main Ideas

Hypothesis versus argument

Bostrom begins by saying the simulation hypothesis is "meant to be understood in a literal sense": an advanced civilization built computers, and our world, including our brains, exists because one of them is running certain programs. He does not think exotic hardware is required. Computers like today's would do if they were much bigger, with more memory and processing power.

The simulation argument is a different thing, and he says people often confuse the two. It does not claim we are simulated. It claims that "at least one of these three is true", without saying which. First, almost all civilizations at our stage go extinct before reaching technological maturity. Second, almost all mature civilizations lose interest in running simulations of their ancestors. Third, we are almost certainly living in a simulation.

Technological maturity

To make the argument work, Bostrom needs only a lower bound on what an advanced civilization could do. He points to Eric Drexler's work on molecular manufacturing, which analyzed machines we cannot yet build but that physics allows, including a computer the size of a sugar cube performing at a million times the human brain. Combine that with space colonization and you get a very high lower bound on available computing power.

His own guess is that there is a ceiling to basic technological capability, and that a civilization might approach it fairly soon after developing machine superintelligence. For the argument itself, though, it does not matter whether maturity takes 100 years or 50 million. Timescales only shift the odds among the three alternatives: the further away maturity is, the more chances there are to go extinct first.

The first two alternatives

For the first alternative to hold, it is not enough that something might destroy us. Nearly every civilization anywhere would have to fail in the same way, which calls for a risk that is close to uniform across all of them. Bostrom notes that the thing most likely to kill us may differ from the thing that would kill almost everyone. He also observes that the closer we get to maturity ourselves, the less probable the first alternative becomes. And being wiped out by a superintelligent AI would not count toward it, because the AI could still reach maturity.

The second alternative requires striking convergence. Since a mature civilization could run vast numbers of simulations with a sliver of its resources, even a small fraction choosing to do so would produce millions. So nearly none of them must want to. Bostrom takes this possibility seriously. He thinks we understand little about the motivations of such civilizations. Once beings can directly choose their own mental states, many of today's instrumental goals may drop away, and considerations we are currently oblivious to may come into view. Lex admits that both of the first two feel unlikely to him, which pushes his intuition toward the third. Bostrom replies that this is not necessarily wrong, but declines to call the three equally probable or to assign numbers.

Consciousness and rendering a world

The argument borrows a premise from the philosophy of mind: that consciousness depends on the structure of a computation and not on biological neurons. Bostrom is confident that a simulation matching a human brain neuron for neuron, with 100 billion neurons wired the same way, would be conscious. He is less sure how much could be simplified away.

A simulation also need not compute everything. As in video games, detail could be rendered only when someone looks, with the atoms of a microphone left as a coarse approximation. He offers dreams as evidence that convincing worlds are cheap. If "three pound lumps of neural matter" can produce immersive realities each night that we fail to recognize as dreams, a civilization with planet-sized computers could do far better.

Lex suspects consciousness is easy to fake, and says he programmed Roomba vacuums to scream and found they seemed conscious to him quickly. Bostrom separates the cases. A brief exchange is easy to fake. But a rich, open-ended relationship that survives probing from every angle might be impossible to implement without actually creating a second conscious mind.

The experience machine

Bostrom introduces Robert Nozick's thought experiment: a machine that gives you any experiences you want for life while you float in a tank. Nozick believed most people would refuse, which suggests we value more than experience. Bostrom lists candidates, such as real connection with other people and having an actual effect on history.

He then adds a twist about status quo bias. Suppose you were told your life so far has been the machine, and you could unplug to become a peanut farmer in Peru. Many would choose to stay with the life they know. So perhaps what we care about is continuing our projects, relationships and identity, wherever they happen to be running. Lex says he would take the leap, and wonders whether the distinction between real and illusion holds up at all. Bostrom's view is that everything in a simulation is real in an important sense, and yet inside and outside remain different situations, and whether the difference matters depends on your values.

Why the third alternative points at us

If the first two alternatives are false, then over the whole history of the universe there would be far more simulated people with experiences like ours than unsimulated ones. To get from there to a claim about yourself, Bostrom uses what he calls the "bland principle of indifference". If you cannot tell from the inside which group you belong to, your confidence should match the proportions. To support it, he asks us to imagine the simulated fraction rising from 90 percent to 99 percent to everyone. At the limit you would be certain, so your credence should approach certainty smoothly along the way.

Simulations could be stacked inside simulations, but everything must ultimately be paid for by the computer in what he calls basement reality, a phrase Lex loves. Given finite resources there, the tower cannot be very tall, and most minds would sit low in it.

The doomsday argument

Lex asks about a related puzzle. Bostrom explains it with two urns, one holding 10 numbered balls and one holding a million. Draw ball number seven and you should become nearly certain you have the small urn. The doomsday argument, discovered by Brandon Carter and developed by John Leslie, applies this to humanity. You are roughly the 100 billionth human. If you treat that as a random draw, a total of 200 billion humans makes your position ordinary, while 200 trillion makes it bizarrely early. So you should shift toward doom coming sooner.

Most people think something is wrong with this, and Bostrom says, "My own view is it seems inconclusive." The trouble is that its key premise, which he calls the self-sampling assumption, seems necessary elsewhere. In a universe large enough that every possible observation gets made by someone, we need something like it to say that our measurements count as evidence at all. His point for the simulation argument is that it relies on a much weaker and less controversial assumption than the doomsday argument does.

Elon Musk, and what is outside

Asked about Elon Musk's interest, Bostrom offers a playful extra consideration. If some simulations include only a subset of people, they would probably favor unusually consequential lives, so someone like Musk has an additional reason to wonder.

On what lies outside, Bostrom thinks we may understand some aspects, the way a dog has some idea of what a human is. He offers an analogy. Suppose the right way to live depended on whether a ten-digit number was odd or even, and we knew the first seven digits. We would know a lot and still have no idea which way to go. He suspects we are in a similar predicament, with the missing pieces being ones that could change the whole picture.

Superintelligence and a good future

Bostrom defines intelligence loosely as the ability to solve complex problems, learn, plan and reason, and thinks a system could be quite intelligent without being conscious. A superintelligence would not need a body. Typing text on a screen would be ample influence. He distinguishes speed superintelligence, for which the evidence is strong, from the qualitative kind, which he finds plausible on looser grounds such as the gap between Einstein and an average person.

He stresses that his 2014 book focused on pitfalls because they were neglected then, and that now the upside deserves attention. He calls artificial general intelligence "the ultimate general purpose technology", useful wherever human problem-solving is, though it would not automatically resolve political conflict. He also wants near-term and long-term AI discussions kept apart: "we can mix them together and have one bad conversation". He expects some form of intelligence explosion with more probability than the average AI researcher, sees no reason for a ceiling near human level, and denies that being outsmarted must mean losing control, which is what alignment research is for. Lex argues that a self-learning system like AlphaZero feels more intelligent than Deep Blue, and Bostrom agrees that general learning ability is an important facet.

For utopia, Bostrom says such a world would force a rethink of values from first principles, and that with resources so expanded we should aim "to do well by the lights of more than one value system" before deciding which to sacrifice. On meaning, he notes that happiness and meaning can come apart, and hopes radical abundance leaves room for many values at once.

Conclusion

The lasting value of this episode is clarity. Bostrom shows that the simulation argument is a three-way choice with no stated winner, and that each branch says something large about our future: that civilizations rarely survive, that survivors want very different things than we imagine, or that we are probably simulated. Around that core he gives accessible accounts of the experience machine, the doomsday argument and the limits of what creatures inside a system can know about it. On AI he is more hopeful than his reputation suggests, while Lex closes with Bostrom's own warning that "Our approach to existential risks cannot be one of trial and error." The full conversation is worth hearing for the care with which each step is taken.

This is a summary of "#83 – Nick Bostrom: Simulation and Superintelligence" from Lex Fridman Podcast, giving you the GIST so you can decide if you want the REST. Listen to the full episode.