Rewrites, forks and breaking changes: hard lessons from language and codec builders
Guido van Rossum weighs what a possible Python 4 could break, Chris Lattner designs Mojo around compatibility after the lessons of Swift and Julia, and the FFmpeg maintainers recount the Libav fork and the push for Rust rewrites. Each has lived through the cost of breaking with what users already have, and their caution is useful to anyone tempted to start over.
The cost of starting over lands on the people who depend on you, and it is almost always larger than the plan assumed. Three sets of builders who have each lived through a break with their users reach that conclusion. None says nothing should ever change. They say change should be additive, opt-in, announced years ahead and held to the standard the old thing already meets.
Guido van Rossum, the creator of Python, speaks from the move from Python 2 to 3. He says the core team "underestimated the pain that transition caused for our users", although they thought they had a good plan. Any Python 4 would need a different reason and a different process. His own sketch, which he calls a personal opinion, is a version without the global interpreter lock. Nothing would change for people writing Python, but the scientific and machine learning world built on C extensions would feel it, and some of those extensions are already short of maintainers. So he imagines years of warning with the new interpreter first offered as an option. Changes that did ship followed the same habit. Type hints reused syntax the language already had, and a recent speedup came from inside the interpreter.
Chris Lattner built the Mojo language as a superset of Python. His lesson came from an earlier compiler project, where real-world code depended on another compiler's quirks. He concluded that it is far easier for a few engineers to implement long-tail compatibility than to persuade every programmer to rewrite. Mojo imports existing Python packages and lets code move across a piece at a time. After a secret, buggy launch of an earlier language, he released Mojo openly as an unfinished version with a roadmap. He says Python is only "not done yet".
Jean-Baptiste Kempf and Kieran Kunhya, who maintain the video software VLC and FFmpeg, address the push to rewrite in the Rust language. Kempf uses Rust for a new company and calls it excellent for new projects, but warns against rewrites because reading code is far harder than writing it. Kunhya says a replacement has to be as good as or better than what exists, and must meet the same testing and stability bar.
They agree that the people who pay for a break are not the people who decide on it, and that compatibility is much of why their projects won. They disagree on how far to go for speed, whether declared types should be enforced, the interpreter lock, and Rust itself. Mojo is new, so its strategy is a design choice and not yet a proven result, and no advocate of a rewrite speaks here.
What to do with it: separate what you would choose from scratch from what you can change now. Find the hidden layer of dependents before promising nothing breaks. Do the compatibility work yourself. Reuse what exists before adding anything. Offer new behavior as an option years before making it the default. Give users a path that works one piece at a time. Launch early and label it unfinished. Hold a replacement to the old thing's tests. Use the new language for new projects.