Will scaling LLMs reach AGI? LeCun versus Amodei and Altman

The Short Version

Yann LeCun, Meta's Chief AI Scientist, argues that autoregressive large language models are missing four capacities intelligence needs: understanding the physical world, persistent memory, reasoning and planning. He says scaling them will not produce human-level AI, which he thinks is at least a decade away and will need new architectures that learn world models from video [1]. Anthropic's Dario Amodei has watched scaling beat every objection since 2014 and, with hedges, eyeballs "powerful AI" around 2026 or 2027 [2]. OpenAI's Sam Altman says GPT-4 "kind of sucks" next to what's coming and expects remarkable systems by the end of the decade [3]. The sides are closer than the headlines suggest. LeCun agrees that scale is necessary [1], Amodei allows that an unexplained plateau could force new architectures [2], and Altman expects a "new paradigm" for slower reasoning [3]. The newest evidence, reasoning models trained with reinforcement learning, fits both sides [4]. Which camp is right determines how fast capabilities arrive, how much the huge compute bets pay off, and how urgent the near-term safety warnings are.

Table of contents

What They Said

Yann LeCun: LLMs are useful, but a dead end for human-level AI

Four missing capacities. LeCun names four features of intelligent behavior in humans and animals: understanding the physical world, persistent memory, the ability to reason and the ability to plan. LLMs, he says, can do none of these, or only in a very primitive way [1]. He is careful to say this doesn't make them useless, since a whole ecosystem of applications can be built on them. But as a route to human-level intelligence, they are missing essential parts [1].

The data gap. His main argument is a comparison of data. LLMs train on roughly 10 to the 13 tokens, about 2 times 10 to the 13 bytes, which would take a person 170,000 years to read at eight hours a day [1]. A four-year-old has been awake about 16,000 hours, and with the optic nerve carrying about 20 megabytes per second, roughly 10 to the 15 bytes have reached the child's visual cortex [1]. His conclusion is that most of what we learn comes from observing and interacting with the world, not from language [1]. Lex Fridman pushed back that language is highly compressed and full of wisdom. LeCun agreed it is compressed, but said that makes it less redundant, and self-supervised learning captures more structure from redundant data like vision [1]. He adds that language only approximates our mental models, and that much of our thinking, like imagining a bottle rotated 90 degrees, has nothing to do with language [1].

Moravec's paradox. Things that seem hard to humans, like chess or the bar exam, are easy for computers, while things we take for granted are hard [1]. LLMs can pass the bar exam, "But then they can't learn to drive in 20 hours like any 17 year old," nor clear a dinner table and load a dishwasher, which any 10-year-old learns in one shot [1].

No planning, compounding errors. An LLM predicts one token at a time, sampling and feeding each back in, which is what "autoregressive" means [1]. Humans, LeCun argues, plan what they'll say at an abstract level largely independent of language. He says his thinking is much the same in French as in English, though his puns are better in French [1]. Because each token carries some chance of leaving the set of reasonable answers, and if those errors are independent (a strong assumption, he notes), the probability of staying correct shrinks exponentially with answer length [1]. Fine-tuning might cover 80 percent of common questions, but the space of prompts has a very long tail. He says the fact that random strings or a few words swapped into another language can derail a model shows it is essentially a giant lookup table [1]. LLMs also spend the same compute per token whether a question is trivial or impossible, so they have fast, automatic System 1 but no deliberate System 2 planning with a world model [1]. And on the Turing test: "we're fooled by their fluency" [1].

Why just add video fails. The next-token trick doesn't transfer to video, he says, because predicting distributions over high-dimensional continuous spaces isn't something anyone knows how to do usefully. No system could predict the carpet texture as a camera pans a room [1]. He says his team at FAIR spent about ten years on generative approaches (GANs, VAEs, masked autoencoders), and that reconstructing corrupted images or video "has been essentially a complete failure" at producing good general representations [1].

His alternative. LeCun is betting on joint embedding predictive architectures (JEPA), which predict in an abstract representation space instead of reconstructing every pixel, keeping what's predictable and discarding noise like leaves blowing in the wind [1]. V-JEPA is the first system his team has built that learns good video representations, and early results suggest it can tell physically possible from impossible videos [1]. With a world model that predicts the next state given an action, a system can plan using model predictive control, a method used for rocket trajectories since the early 1960s and one LLMs cannot perform [1]. Future dialogue systems, he says, should "think about their answer, plan their answer by optimization before turning it into text" using energy-based models [1]. Hierarchical planning, like breaking a trip from NYU to Paris down to standing up from a chair, remains unsolved: "Nobody really knows how to do this in AI" [1]. His bottom line: "If you're really interested in human-level AI, abandon the idea of generative AI" [1]. He also warns that combining vision with language too early is "cheating," using language as a crutch, since cats and dogs understand the world far better than any LLM without language [1].

On scale and timelines. LeCun doesn't dismiss compute: "scale is necessary but not sufficient" [1]. He notes a GPU draws half a kilowatt to a kilowatt against the brain's roughly 25 watts, so new hardware principles may be needed [1]. He thinks human-level AI is at least a decade away and probably much more, and "it's not going to be an event" but gradual progress [1]. He treats intelligence as a multidimensional set of skills rather than a single number [1].

Credit where due. LeCun explains why LLMs are so impressive: self-supervised learning, which he has long championed [1]. The surprise around GPT-2 was that decoder-only models kept understanding more as they scaled [1]. His objection is to scaling LLMs as the route to human-level intelligence, not to scaling in general.

Dario Amodei: scaling keeps answering its critics

How he became convinced. Amodei traces his belief to late 2014, working on speech recognition at Baidu with Andrew Ng. While others said the field lacked the right algorithms, he treated model size, data and training time as independent dials and saw performance keep rising as each went up [2]. GPT-1 in 2017 convinced him language could go furthest, because there are trillions of words to train on [2]. He credits Ilya Sutskever, Rich Sutton's bitter lesson and Gwern's writing, and recalls Ilya telling him, "the thing you need to understand about these models is they just want to learn" [2].

The pattern of objections. At every stage, Amodei says, critics raised a new wall: syntax but not semantics, sentences but not paragraphs, running out of data, models can't reason. Each time, scaling either found a way around or was the way around [2]. He compares scaling to a chemical reaction in which size, data and compute must rise together or the reaction stops [2]. As a former physicist, he explains why bigger is better through 1/f noise and long-tailed distributions. Small networks capture common patterns like frequent words and basic grammar, while larger ones pick up progressively rarer and more complex patterns, from sentences to paragraphs to themes [2]. He sees no ceiling below human level and suspects fields like biology have "a lot of room at the top" [2].

Evidence he cites. On SWE-bench, real software engineering tasks, the state of the art went from roughly 3 percent at the start of the year to about 50 percent with the new Sonnet 3.5, and he guesses 90 percent within a year [2]. He describes model ability climbing from high school to undergraduate to PhD level [2]. He estimates frontier models cost around $1 billion, rising to a few billion next year and above $10 billion in 2026, with ambitions for $100 billion clusters by 2027 [2].

What could stop it. Amodei is not dogmatic. He lists possible limits: running out of high-quality data (though synthetic data, self-play like AlphaGo Zero, and reasoning models trained with reinforcement learning may get around it), an unexplained plateau requiring new architectures, or compute costs [2]. He says scaling laws are empirical regularities, not laws of the universe [2]. He adds that RLHF mainly bridges the gap between model and human rather than making models smarter, while other forms of reinforcement learning may genuinely improve reasoning. Post-training may even come to dominate costs [2].

Timelines, with hedges. He dislikes the term AGI, comparing it to treating "supercomputers" as a discrete threshold when progress is a smooth exponential [2]. His "powerful AI" is smarter than a Nobel laureate across most fields, uses every modality, works autonomously for hours to weeks and runs as millions of copies at 10 to 100 times human speed [2]. He stresses that this isn't a scientific prediction, but eyeballing the trend suggests powerful AI around 2026 or 2027, with derailers like data limits or a disruption to chip production [2]. He rejects both a days-long singularity, which ignores physical limits and human institutions, and the view that gains will be as underwhelming as the computer revolution. He expects major change over five to ten years rather than fifty to a hundred [2]. "AI is not God," he adds, even as he imagines compressing progress expected by 2100 into roughly 2027 to 2032 [2].

Why it matters for risk. Because models improve so fast, Amodei says catastrophic risks are "like ghosts, but they're coming at us so fast" [2]. He told the Senate that serious bio risks might arrive in two to three years, and his Responsible Scaling Policy ties stricter safeguards to capability thresholds [2]. He would not be surprised if ASL-3, models that meaningfully help non-state actors, arrives next year, and would be very surprised if it took until 2030 [2]. He calls these risks "not sci-fi fantasies" [2]. His safety program depends on his scaling forecast. If capability jumps keep coming, the safeguards have to be in place before them.

Sam Altman: today's models will look primitive

Relative to the future, GPT-4 is weak. Asked what impresses him most about GPT-4, Altman says, "I think it kind of sucks" [3]. GPT-3 now looks unimaginably horrible, and he expects the gap from GPT-4 to GPT-5 to match the gap from GPT-3 to GPT-4 [3]. His job, he says, is to live a few years in the future [3]. He describes the next model as getting better across the board [3]. He attributes progress to a line he paraphrases from Ilya about multiplying 200 medium-sized things together into one giant thing [3]. That's a picture of many improvements compounding, not a single scale knob.

Models understand more than critics admit. Answering LeCun-style doubts about world models, Altman says, "I think all of these models understand something more about the world model than most of us give them credit for" [3]. Because they fail at some clear things, "it's easy to like look at the weaknesses see through the veil and say this is just this is all fake but it's not all fake it's just some of it works and some of it doesn't work" [3]. He points to early Sora videos where a person walks in front of an object and "the same thing was still there," and says "fundamentally these models are just getting better and that will keep happening" [3]. He won't claim Sora has a great underlying 3D model, and he notes failures like cats sprouting extra limbs will improve with scale but may also reflect something fundamentally different from human learning [3].

He also sees a missing piece. Altman agrees with LeCun's point about equal compute per token. You want an AI "to be able to think harder about a harder problem and answer more quickly about an easier problem" [3]. "I think there will be a new paradigm for that kind of thinking," he says, though whether it's a layer on top of LLMs or something else is less important to him [3]. On Q*, he says only that better reasoning is an important direction OpenAI hasn't cracked yet [3].

Compute as currency. "compute is going to be the currency of the future," he says, and demand for intelligence behaves like demand for energy: cheap enough, and the world will use enormous amounts [3]. Energy is the hardest part, followed by data centers, supply chains and fabs [3].

AGI as a mile marker. Altman calls the question of when AGI arrives poorly formed, because definitions vary so much. "AGI is also not an ending," he says, but more of a mile marker [3]. He still expects quite capable, remarkable systems by the end of the decade and possibly sooner [3]. His test is whether a system significantly increases the rate of scientific discovery. He notes that the global economy doesn't feel different since GPT-4 [3].

Risk framing. Altman lays out a quadrant of short or long timelines and slow or fast takeoff: "short timelines slow takeoff is the safest quadrant" [3]. He defends iterative deployment because "AI and surprise don't go together" [3]. Existential risk from the AI itself isn't currently his top worry. "It's not super easy to escape the box at this time," and fixating on that "theatrical risk" crowds out others, though he says OpenAI works hard on it and could be surprised [3].

The industry view: Dylan Patel and Nathan Lambert

Semiconductor analyst Dylan Patel and Ai2 researcher Nathan Lambert add evidence from outside the labs [4]. Their most relevant story is DeepSeek R1-Zero. Large-scale reinforcement learning on verifiable problems, applied directly to a base model with no human preference data, produced reasoning behaviors like "wait, let me check this" on their own [4]. Fridman reads Andrej Karpathy's view that learning by trial and error is more powerful than imitation: "two is significantly more powerful" and "the source of all magic is always two" [4]. Patel says there has been "a 1200 x reduction in cost to achieve the same level of intelligence as GPT-3" in about three years [4]. He notes SWE-bench scores rising from about 4 percent to 60 percent in a year [4].

They are also skeptical. Lambert says, "For me, it's probably after 2030," and discounts AI CEOs' two-year forecasts because of their incentives [4]. Patel expects capabilities around 2027 or 2028 but says the cost of running them at scale will slow deployment [4]. On agents, Patel compares chaining tasks to semiconductor yields: even high reliability per step compounds into failure over many steps [4]. That is the same arithmetic as LeCun's argument about compounding errors [1]. Lambert connects DeepSeek's engineering tricks to Sutton's bitter lesson, which favors simple, scalable methods over clever human priors, and suggests some of those tricks may matter less over time [4]. He also stresses that data quality is "the number one determinant of the model quality" [4]. Scale alone is not the whole story even inside the pro-scaling camp.

Where They Agree, and Where They Don't

What they agree on. All four sources agree that current LLMs are useful and improving [1][2][3][4]. LeCun calls them a foundation for a whole ecosystem of applications [1]. All agree that compute matters. LeCun says scale is "necessary" [1], Amodei's strategy depends on it [2], Altman calls compute the future's currency [3], and Patel documents gigawatt-scale buildouts [4]. And all agree today's systems lack good deliberate reasoning. LeCun says LLMs have no System 2 [1], Altman wants a new paradigm for slower thinking [3], and Amodei and Lambert point to reinforcement learning for reasoning [2][4].

The real disagreement: is the missing piece inside or outside the paradigm? LeCun's view is that the gaps are architectural. Token prediction over text cannot yield physical understanding, persistent memory or hierarchical planning, so a new paradigm (JEPA, energy-based planning) is needed [1]. Amodei's view is that every previous "missing piece" was eventually covered by scaling or by methods that ride on top of it, such as synthetic data, self-play and reinforcement learning [2]. Altman sits in between. He believes models already understand more than critics credit and will keep improving, but he expects a new paradigm for reasoning, without saying whether it will be built on LLMs or not [3].

Why each side believes what it does. LeCun reasons from first principles and from his own failures. He points to the data gap between text and vision, a decade of failed generative video work at FAIR, the math of compounding errors, and Moravec's paradox [1]. Amodei reasons from a track record. He has seen a decade of curves going up and a series of predicted walls that didn't hold [2]. Altman reasons from inside the product cycle, where each model makes the last look primitive [3]. Each argument has a weak point. LeCun's compounding-error math rests on an independence assumption he calls strong [1]. Amodei concedes scaling laws aren't laws of nature [2]. And Altman notes that GPT-4 hasn't yet changed the global economy [3].

Timelines. Amodei's rough estimate is 2026 to 2027 for powerful AI [2]. Altman expects remarkable systems by the end of the decade, maybe sooner [3]. Patel says 2027 or 2028 for capabilities, with slower deployment [4]. Lambert says after 2030 [4]. LeCun says at least a decade and probably much more [1]. Definitions differ too. Amodei and Altman both reject AGI as a single threshold [2][3], and LeCun says it won't be an event [1]. Some of the gap in dates is about what's being measured, not only what will happen.

Risk. LeCun dismisses doomer scenarios because progress will be gradual and correctable, because intelligence doesn't imply a drive to dominate, and because "my good AI against your bad AI" will filter misuse [1]. Amodei takes near-term catastrophic misuse seriously because he expects capability to arrive fast [2]. Altman wants a slow takeoff and warns that "theatrical" risks distract from subtler ones [3]. Timelines drive risk views directly. If LeCun is right that progress will be slow and architectural, there is time to iterate guardrails the way jet engines became reliable [1]. If Amodei is right, safeguards must be ready before capabilities arrive [2].

Incentives. Lambert explicitly discounts CEO forecasts for their incentives [4]. That applies to Amodei and Altman, who raise capital on scaling. LeCun's employer, by his account, earns money from ads and business customers and releases its models openly [1], so it depends less on a fast AGI story. Incentives don't settle who is right, but readers should weigh them.

Why It Works

Each side offers a mechanism.

Why scaling has worked (Amodei). Language data follows long-tailed distributions. More parameters, data and compute let a network capture progressively rarer, higher-level patterns, from grammar to paragraphs to themes [2]. The three must scale together, like reagents in a chemical reaction [2]. LeCun gives the underlying reason from his side: self-supervised learning, filling in masked text in giant networks, yields broad capabilities, and decoder-only models kept understanding more as they grew [1].

Why scaling LLMs may stall (LeCun). Text contains far less raw information than sensory experience [1]. Autoregressive generation lets errors compound, and fine-tuning can't cover the long tail [1]. Fixed compute per token rules out deliberate System 2 search [1]. Predicting pixels fails because distributions over continuous high-dimensional spaces can't be represented usefully. Planning requires a world model that predicts the consequences of actions, which LLMs lack [1].

How the gap might be bridged. Reinforcement learning on verifiable problems is the clearest candidate. In DeepSeek R1-Zero it made self-checking behavior emerge [4]. Amodei says it may genuinely make models reason better [2], and it responds to both LeCun's and Altman's complaint about equal compute per token [1][3]. OpenAI's o3 on ARC-AGI ran many chains of thought in parallel and selected answers, a form of search [4]. At the same time, reasoning multiplies inference cost. Tens of thousands of output tokens swell memory use, and o3 cost an estimated five to twenty dollars per ARC-AGI question [4]. That bears on Patel's point that deployment could lag capability [4].

Cost curves. Patel's 1,200-fold drop in the cost of GPT-3-level intelligence [4] and Altman's energy analogy for demand [3] explain why investment keeps flowing: every capability level gets cheap quickly. LeCun's 25-watt brain comparison [1] suggests efficiency, not just size, may be what ultimately limits progress.

Where It Applies, and Where It Doesn't

Where scaling has the strongest case: verifiable, text-and-code domains. Programming benchmarks are rising fast [2][4]. Amodei expects programming to change fastest because models can write, run and check code in a closed loop [2], and Patel expects software engineering to be the first domain where agents succeed because code is verifiable [4].

Where LeCun's critique bites hardest: the physical world. Driving, household tasks and robotics depend on world models and low-level planning [1]. LeCun says even FAIR's robot demos fall far short of clearing a dinner table [1]. Altman concedes it would be depressing to have AGI yet need humans for everything physical, and says OpenAI will return to robotics [3]. Fridman is skeptical of fast progress in robotics [4]. Patel, notably, expects language models' big breakthrough to come from computer use and robotics via self-play in verifiable environments [4], which cuts the other way.

Long, multi-step tasks. LeCun's compounding-error math [1] and Patel's yield analogy for agents [4] both predict trouble with long chains. Amodei expects computer use to climb toward human-level reliability on the same trajectory as coding [2]. This is a live test of the two views.

Where it matters for decisions. Investment, energy buildouts and export controls all assume a timeline. Patel argues export controls only make sense if AI changes society within five to ten years; otherwise they help China in the long run [4]. Amodei's safety thresholds and regulatory urgency assume capability is coming fast [2]. LeCun's call for open source and against restrictive regulation assumes gradual progress that can be corrected [1].

What no one can settle yet. Amodei calls his timeline an eyeballed extrapolation, not a prediction [2]. Altman won't say what Q* is [3]. LeCun's JEPA world models are still at early stages, like telling whether a video is physically possible [1]. Neither side has a decisive result.

What To Do With It

1. Sort claims into two types. Separate more compute will help (which LeCun accepts) from more compute on this architecture is enough (which he rejects) [1][2]. Most loose talk mixes them.

2. Discount forecasts for incentives, including your favorite's. Lambert's advice to discount CEO timelines [4] applies to the companies raising money on scaling. Also consider that LeCun's own lab has a competing architecture to promote [1].

3. Watch coding benchmarks as the scaling camp's test. If SWE-bench-style scores keep rising toward Amodei's predicted 90 percent [2], and agents handle longer tasks, the scaling view gains ground. If they plateau, the scaling view looks weaker.

4. Watch physical-world tasks as LeCun's test. Progress on robots that can clear a table [1], or JEPA-style models that learn physics from video, would support his path. Robust physical understanding emerging from scaled models trained on video would support Altman's view [3].

5. Track reasoning cost, not just reasoning ability. Inference for reasoning models is expensive [4]. Check whether cost curves keep falling, as Patel documents [4], before assuming capabilities will be deployed widely.

6. Treat timelines as ranges. The sources span 2026 [2] to beyond 2034 [1]. Plan for a range and set checkpoints, rather than betting on a single date.

7. Take near-term misuse risks seriously whichever side wins. LeCun doubts LLMs give bioweapon uplift beyond a search engine [1], while Amodei builds safeguards against it [2]. Guardrails, iterative deployment [3] and published model behavior specs [3] are low-regret steps under both views.

8. If you're a researcher or student, look at the open problems. LeCun points to world models from observation, planning with learned models and hierarchical planning, where he says there has been zero demonstration of learned multi-level action representations [1]. Amodei points to interpretability, long-horizon tasks and multi-agent work [2]. Both lists offer important problems that don't need a frontier-lab budget.

Sources

1. Yann LeCun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI · Lex Fridman Podcast
2. Dario Amodei: Anthropic CEO on Claude, AGI, the Future of AI, and Humanity · Lex Fridman Podcast
3. Sam Altman: OpenAI, GPT-5, Sora, Board Saga, Elon Musk, Ilya, Power & AGI · Lex Fridman Podcast
4. DeepSeek, China, OpenAI, NVIDIA, xAI, TSMC, Stargate, and AI Megaclusters · Lex Fridman Podcast