Why self-driving is hard: the problem is other people
October 5, 2026 · 14 min read
Anca Dragan, Michael Littman, Andrej Karpathy, George Hotz and Lex Fridman all locate the difficulty of autonomous driving in predicting and influencing human behavior and in rare edge cases, more than in seeing the road. It explains why full autonomy keeps slipping and why partial automation creates its own trap for drivers.
The rules, numbers and steps worth keeping in front of you.
- The hard part of driving is other road users and rare situations, not seeing lanes and objects.
- Dragan: remove every human from downtown San Francisco and driving is done.
- Dragan: people are messy and irrational, so robots' rational models mispredict them.
- Dragan: a car's actions change what others do; a planner that assumes traffic will not react gets stuck at a merge.
- Littman: driving is a social activity that needs thinking about what other drivers are thinking.
- Karpathy: the data engine finds rare failures, adds them to training and retrains.
- Chollet: L4 is probably achievable, L5 is probably not, by pure learning.
- Darling: robot cars are great at predictable things; the handoff to the person is a huge unsolved problem.
- Fridman: what is sold as full self-driving is semi-autonomous and needs human supervision.
- Dragan's warning: do not assume a supervising human is as safe as a driving one.
- Sensors are disputed: Dragan wants lidar where available; Karpathy says vision is sufficient.
- These conversations are several years old and do not describe any current product.
- Follow the owner's manual, check local rules, and stay ready to drive.
This is the short form. The full insight has every expert's reasoning, their own words and the sources.