Software 2.0: when learned models replace hand-written rules
Andrej Karpathy describes Software 2.0 and Tesla's data engine, George Hotz explains comma's learned driving policy, and the DeepSeek analysts describe reinforcement learning's aha moment. Each shows systems improving when engineers stop writing rules and let data and training do the work.
Software 2.0 is the shift from writing a system's behavior by hand to supplying data, an objective and training, and letting the behavior be learned. In vision, driving and language models the systems got better when engineers made that switch. The engineering does not go away. It moves to data quality, reward design, verification and the infrastructure that makes training possible.
Andrej Karpathy, the former director of AI at Tesla, named the idea. Code is increasingly written in the weights of a neural network, programming becomes curating datasets and objectives, and training acts as the compiler. At Tesla, Autopilot began as mostly hand-written code and networks took over the harder jobs. The lever is data that is very large, accurate and diverse, and the loop he calls the data engine: deploy the model, find the rare cases where it struggles, establish the truth for them, add them to training and retrain. He says the hard part is execution, not ideas, and that "10x problems are not 10x hard" because an ambitious goal forces a new approach. He also removed parts such as extra sensors and detailed maps, since each adds code and noise.
George Hotz, the founder of comma.ai, states the driving problem as "how do you build a model that outputs a human policy for driving?" His answer is to learn it from data instead of hand-coding it, then let the system practice inside a learned simulator. His reward is not how human the driving looks but whether a person would take over.
Dylan Patel and Nathan Lambert describe DeepSeek's reasoning model. Reinforcement learning on problems with checkable answers, such as math and code with tests, was applied to a pretrained base model, and reasoning habits like checking its own work emerged without anyone writing them in. Lambert adds that data filtering and quality are the number one determinant of model quality.
All three agree on the direction, that data is where the work goes, and that ideas are not the bottleneck. They differ on how far to go. Tesla teaches from labeled truth, Hotz adds trial and error, and the DeepSeek and AlphaZero examples improve by removing human data. That works where a machine can check the answer, which driving lacks. Trial and error from a blank start failed in an early Karpathy project, so the pretrained starting point matters. On AI-written code, Karpathy warns that his coding assistant can introduce subtle bugs and must be supervised, Hotz wanted a bug spotter, and Patel expects coding agents to succeed first.
The approach fits less well when nothing verifies the answer, when a task has many chained steps, when subtle errors are costly, when the data carries unwanted bias, or when you cannot afford the fleets, labelers and failed runs. Hand-written rules still surround learned models.
What to do with it: before adding more special cases, ask whether the behavior can be learned. Treat the dataset as the source code. Build the loop, not just the model. Give people and machines the labeling jobs each does well. Choose a reward that matches what users care about. Start from a pretrained model and add trial and error where answers can be checked. Remove parts that do not earn their place. Supervise and test the output, and use the system yourself.