Why some scientists doubt AI doom: four arguments from skeptics
Michael Littman, Lee Cronin, Stephen Wolfram and Michael I. Jordan give technical reasons to doubt a sudden hostile superintelligence, from the limits of self-improvement to the absence of a single apex intelligence. Their arguments let readers test the doom case against specific objections.
Five researchers who build or study AI systems give four reasons to doubt the doom scenario, the story in which machines design smarter machines until they pass us and pursue goals out of step with ours. It will not happen suddenly. Self-improvement has limits. Intelligence has no single top. And being intelligent does not bring a wish to dominate. None of this shows AI is harmless, and the same people name other dangers.
The first argument is that it will not spring into being. Michael Littman, a computer scientist at Brown, is not moved by the idea of accidentally creating a superintelligence, because building such systems teaches us how they work and keeps us in the loop. Yann LeCun, Meta's chief AI scientist, puts human-level AI at least a decade away and says "it's not going to be an event". Intelligence will be added gradually, with guardrails. Michael I. Jordan of Berkeley says thinking machines are not what this era is building at all.
The second is limits. Littman notes that a Go program gets better at Go and nothing else, that finite games have an optimal strategy, and that apparent exponentials may be the front half of an S-curve. LeCun says language models lack understanding of the physical world, persistent memory, reasoning and planning, and that scale alone will not supply them. Lee Cronin, a chemist, says current models interpolate from the past. Stephen Wolfram adds that for many systems the only way to know what they will do is to run every step, so even a superintelligence must wait.
The third is that there is no ladder. Wolfram doubts any apex intelligence exists. LeCun calls intelligence a collection of skills, and Jordan points out that markets are intelligent without being agents.
The fourth is will. Cronin says "the AI doom scenario is nonsensical in the extreme" because no mechanism for it is known. LeCun says the drive to dominate evolved in social species and need not be built into machines. Wolfram says an AI has no intrinsic idea of what goals to pursue.
They disagree with each other. LeCun expects human-level AI and doubts only the catastrophe, while Cronin doubts the capability itself. Wolfram admits it felt a little scary to let a model run code on his own computer and says no sandbox is perfect. The doom side answers only through host Lex Fridman, whose points that physics does not forbid it and that we keep handing over infrastructure are set aside, not refuted. Several arguments rest on today's architectures and may not last.
What they do worry about is social media algorithms steering behaviour, fake data and fake users, people sending whatever the machine suggests, control concentrated in a few companies, and hype.
What to do with it: test any doom argument against four questions. Does it need a sudden jump, limitless improvement, a single scale of intelligence, or a machine that wants something? Ask for the mechanism. Do not take fluency for understanding. Watch the present harms. Keep an invested human in consequential decisions, and be careful what you let a model control. Know what would alarm you; for Littman it is a system that resists being shut off. Favour many systems over a single gatekeeper.