Rewrites, forks and breaking changes: hard lessons from language and codec builders

The Short Version

Three sets of builders who have each lived through a break with their users reach the same conclusion: the cost of starting over lands on the people who depend on you, and it is almost always larger than the plan assumed [1][2][3]. Guido van Rossum says the Python core team "underestimated the pain that transition caused for our users" in the move from Python 2 to 3, and that any Python 4 would need both a different reason and a different process [1]. Chris Lattner built Mojo as a superset of Python because, since his Clang days, he has held that it is far easier for a few engineers to implement long-tail compatibility than to persuade every programmer to rewrite their code [2]. On the FFmpeg and VLC side, Jean-Baptiste Kempf calls Rust excellent for new projects but warns against rewrites because reading code is far harder than writing it, and Kieran Kunhya says a replacement has to be as good as or better than what already exists before people will switch [3]. None of them argues that nothing should ever change; they argue that change should be additive, opt-in, announced years ahead and held to the standard the old thing already meets [1][2][3].

Table of contents

What They Said

Guido van Rossum: the transition he would not run the same way twice

Van Rossum's caution starts with small things. Asked whether Python could ever adopt curly braces, he answers that braces already mean something else in Python, mostly dictionaries, and that the language would get in trouble if it tried to switch [1]. He generalises immediately: "You can't change that kind of thing in a language" [1]. He adds that the same is true in the other direction, since C could not be redefined to use indentation even if everyone agreed indentation would be better in a green field [1]. His reason is partly social: "It's hard enough to reach agreement over much more minor details" [1]. In a green field he would still argue for indentation, because it "frees up a pair of matched brackets of which there are never enough in the world for other purposes" [1]. The preference and the constraint are separate questions for him: what would be best from scratch does not tell you what an existing language can do [1].

The Python 3 transition is the experience behind that view. Asked whether there will ever be a Python 4, he says the community and the core developers have "a large amount of painful memories about the Python 3 transition" and that everybody is happy there will be no 4.0, at least not anytime soon [1]. He is specific about what went wrong: the team "underestimated the pain that transition caused for our users" [1]. He does not say the plan was careless. In his words, "we just thought that we had a good plan" [1]. He also says that, had they known, they could have designed Python 3 somewhat differently without making it any worse [1]. His conclusion is that a Python 4 would need "a different reason for having that and a different process for managing the transition" [1].

He then sketches what such a version might be, stressing that "this is definitely a personal opinion" and that not everyone agrees with it [1]. The trigger he imagines is a Python without the global interpreter lock becoming the default [1]. Nothing about the syntax or the semantics of the language would change for people writing Python code [1]. The break would be underneath, for the part of the ecosystem built on C extensions, which he describes as the entire machine learning, data science and scientific Python world [1]. In his words, "those people would likely feel the pain the most" [1]. He notes that there is a large list of extensions that would have to do work, and that some of them are already "low on maintainers" [1]. The process he describes is therefore slow: several years of warning, "maybe five years earlier", with the no-GIL interpreter first offered as an option, possibly one chosen when Python is compiled from source [1]. He is open that the version numbers in his example are invented: "I'm just making the specific numbers up" [1]. Anyone choosing that option would also need matching versions of every extension module they use [1].

The same habit shows in changes that did ship. Type hints arrived because van Rossum and mypy's author Jukka Lehtosalo, who had first invented a syntax that was not quite compatible with Python, settled on a compromise that "would not require any changes to Python" [1]. Van Rossum says the key was reuse: "we already had a syntax for annotations" [1]. The hints are not enforced when code runs, partly because annotations are sometimes wrong and enforcing them would break many dynamic programs [1]. Python 3.11's speedup followed the same pattern: almost all the work went into the interpreter, the team did not build a just-in-time compiler, and the new specialised instructions fall back to the generic path when their guess about types is wrong [1]. Van Rossum explains that he originally wanted something working in "about three months" and says "I sort of cut corners", which left thirty years of low-hanging fruit that could be picked without touching what users write [1].

On the fork in his own field, he is unconvinced. A developer at Facebook has built a no-GIL fork of CPython that uses all cores with limited single-thread slowdown, but van Rossum doubts that enough people need multithreaded speed to justify the complexity and maintenance, and he calls the GIL a pretty nice Goldilocks point between no threads and all threads all the time [1]. The GIL itself, he recalls, exists because the team did not want to rewrite the whole interpreter to be thread safe when operating systems added threading in the early 1990s [1].

Chris Lattner: compatibility as the design brief

Lattner's position comes from a war story about building Clang around 2006 [2]. He intended a pure, standards-compliant C parser, but real-world headers such as stdio.h depended on GCC's quirks [2]. He concluded that it is far easier for engineers to implement long-tail compatibility than to persuade all C programmers to rewrite their code [2]. The same reasoning drives Mojo's design as a superset of Python, and his stated goal is to meet programmers where they are rather than force them to rewrite [2].

In practice that means several things. Mojo can already import arbitrary Python packages through CPython, keeping the objects in CPython's representation, which gives an incremental migration path [2]. Packages that are half Python and half C++ could move to Mojo over time while still exposing Python interfaces, which is how Swift coexisted with Objective-C [2]. The def keyword works exactly as in Python, while fn is a stricter mode for those who want it [2]. Types are adopted progressively, and Lattner rejects the idea that untyped Python is a "guilty pleasure" that must be rewritten for serious work [2]. People have already moved small pieces of Python into Mojo unchanged and seen 12x speedups [2]. Modular's engine is likewise designed to sit underneath TensorFlow and PyTorch rather than replace them, so that it improves performance without forcing rewrites [2].

He is careful about the Python 3 history too. He consulted van Rossum before launch, describes Mojo as a member of the Python family, and politely declines to call it Python 4.0 because of the trauma of the Python 3 transition [2]. His verdict on Python is that it is not broken, only "not done yet" [2].

Two failures inform this. Swift for TensorFlow, his research project at Google, did not succeed, and his explanation begins with the plain fact that Swift is not Python; TensorFlow at the time was also not set up for eager mode [2]. He says he was surprised by the Julia community's reaction to Mojo and that he respects Julia, but that it still would not be Python [2]. The second lesson is about how to launch. Swift was developed in secret for four years, known to about 250 people, and its 2014 debut came with many bugs, surprised Apple's own engineers and saddled the compiler with technical debt [2]. Mojo was therefore released openly as a 0.1, not for production, with a published roadmap [2]. Two weeks in it had over 70,000 sign-ups for its playground and more than 10,000 people on its Discord, and community feedback had already renamed one keyword to inout [2].

Lattner also describes rewriting as a standing cost in his industry rather than a one-off. Large models must be rewritten from Python into C++ to be deployed, a cycle that can take weeks or months and teams of 45 people [2]. Each generation of chips requires hand-tuned rewrites that only companies with hundreds of engineers can afford [2]. Part of Mojo's pitch is that autotuning lets old code adapt to new hardware automatically [2]. He names complexity as the "bitter enemy" behind all of it [2].

Jean-Baptiste Kempf and Kieran Kunhya: Rust, the fork, and what is already in the code

The FFmpeg and VLC maintainers address the push toward Rust directly. Kunhya says he values Rust's memory-safety goals but that the language has an "Esperanto vibe" [3]. He compares it to the Sinclair C5: to get people to switch, something has to be as good as or better than what exists [3]. He would welcome Rust in FFmpeg if it meets the same testing and stability bar [3]. Kempf is not a Rust sceptic, since his startup Kyber is written mostly in Rust, and he calls the language excellent for new projects [3]. His warning is against rewrites specifically, because reading code is far harder than writing it [3]. He adds a technical caveat for their field: pairing Rust with handwritten assembly breaks the safety model, and he suggests compile-time checked assembly as an answer [3].

On the 2011 split between FFmpeg and Libav, the two frame it as a normal open source fork that ended with Libav's work merged back and FFmpeg stronger [3]. The summary of their conversation gives no more detail than that, so the fork's causes and costs cannot be reconstructed from this source [3]. Kempf does treat forks as a real force elsewhere in the conversation: he argues that if he had accepted offers to bundle toolbars or ads into VLC, the result would have been a fork that killed the project within years [3].

The rest of their conversation shows why they guard existing code. FFmpeg's assembly typically runs 10x to 50x faster than C by Kunhya's account, and the dav1d decoder contains roughly 30,000 lines of C and 240,000 lines of handwritten assembly under the motto "every cycle matters" [3]. Both men reject the claim that compilers can match this automatically, saying the gap is multiples, not a few percent [3]. Kempf says this kind of work cannot simply be vibe-coded, and Kunhya notes that current language models struggle with it because there is not enough training data [3].

They have also paid for breaks of their own. Kempf moved libVLC's core from the GPL to the LGPL so that companies could embed it, and because every contributor keeps their copyright he had to track down more than 350 people to do it [3]. He calls a licence "a social contract" with the community [3]. And he recalls receiving a death threat, with fake powder, around 2009 to 2010 for ending PowerPC support [3].

Where They Agree, and Where They Don't

All three conversations agree that the people who pay for a break are not the people who decide on it. Van Rossum points to extension maintainers who are already short-staffed [1]. Lattner points to every C programmer who would have had to rewrite code that worked [2]. Kempf and Kunhya describe a world of volunteer maintainers and burnout, including the xz backdoor, in which a lone maintainer was worn down by attackers [3].

They also agree that compatibility is a large part of why their projects won at all. Van Rossum compares Python's dominance in science to everyone driving on the same side of the road, and describes scientists in different fields exchanging libraries; he asks "Why start from scratch" in a language without that infrastructure [1]. Lattner calls Python "the universal connector" [2]. VLC still runs on everything from Windows XP to Windows 11 and even on OS/2, and it was built from the start to handle broken or incomplete input [3].

The disagreements are real, though.

How far to go for speed. Van Rossum's team made Python 3.11 faster from inside the existing interpreter [1]. Lattner is appreciative but says that work does not run GPUs or vectors, and adds "20% good 35 000 times is better" [2]. Mojo, he says, worked backward from the limits of physics rather than forward from making Python a little faster, on a completely new compiler design he calls "super hard" [2]. The disagreement is about where the new work goes, and both keep the user's Python code intact [1][2]. The two summaries also give different figures for the same release: the van Rossum conversation says 10 to 60 percent faster, while the Lattner conversation says 15 to 20 percent [1][2].

Whether declared types should bind. Van Rossum keeps hints unenforced so that dynamic programs keep running, and he sees the several competing type checkers, which release far more often than Python does, as a laboratory of experiments [1]. Lattner lists the same facts as weaknesses: the hints are ignored by the interpreter, interpreted differently by competing tools, and cannot be used for performance; in Mojo a declared type is enforced by the compiler [2]. Each view follows from the speaker's position: van Rossum has existing programs to protect, while Lattner is adding a stricter layer that nobody is obliged to use [1][2].

The GIL. Van Rossum calls it a Goldilocks point and doubts that enough people need more [1]. Lattner describes CPython's reference counting as tightly intertwined with the lock that hurts parallelism, and counts threads across many cores among Mojo's gains [2].

Rust. Kunhya is cool on the language itself, while Kempf builds a company on it [3]. They meet on the condition: new code in a new language is fine for new projects, and for an existing project it has to clear the bar the existing code already clears [3].

What a fork means. Van Rossum treats the no-GIL fork as a proposal whose maintenance burden has yet to be justified [1]. The FFmpeg maintainers treat their fork as ordinary and, in the end, beneficial [3]. Kempf also describes a fork as the thing that would have killed a compromised VLC [3].

Why It Works

The arithmetic favours the implementer absorbing the cost. Lattner's Clang lesson is a comparison of two quantities: the work a few compiler engineers do to match quirks, against the work of persuading all C programmers to change [2]. Van Rossum's Python 4 sketch is the same sum, with a large list of extensions on the other side [1].

The break usually falls on a layer the designers do not see. A no-GIL Python changes nothing for pure Python code, yet the scientific stack sits on C extensions that would need to be recompiled against a new interface [1]. Clang's parser was correct by the standard and still failed on real headers [2].

Network effects were the reason for success, so breaking them removes the reason. Van Rossum describes libraries being shared between groups doing completely different mathematics, and new frameworks picking Python because it was "already known to our users" [1]. Lattner describes reinforcing growth loops: Python owns machine learning and is taught everywhere [2].

Old code holds knowledge that is expensive to recover. Kempf's argument is that reading code is far harder than writing it [3]. Kunhya's account of reverse engineering shows the extreme case: he estimates one megabyte of binary takes about a month of work, and he describes how a codec is really a collection of approaches because "it grows naturally" [3].

Syntax and promises are one-way doors. Van Rossum notes that once syntax enters the language it cannot easily be removed, which is why experiments happen in type checkers first [1]. Lattner draws the mirror lesson from Swift's secret launch and ships a 0.1 so that mistakes can be corrected before anyone depends on them [2].

Tests define what compatible means. FFmpeg enforces bit-exactness, under which "every implementation must produce exactly the same bits", through its FATE system across a huge matrix of operating systems, compilers and CPU architectures [3]. Kunhya calls the lack of such a definition in MPEG-2 "one of the biggest mistakes the video industry made" [3]. That is the concrete content of his stability bar for Rust [3].

Additive change pays out early. Python's specialising interpreter falls back when wrong, so no program had to change to benefit [1]. Unchanged Python code moved into Mojo ran 12x faster [2].

Where It Applies, and Where It Doesn't

The caution applies most strongly where there is a large installed base and a long tail of dependents maintained by people with little time: Python's C extensions, C's system headers, the world's video files [1][2][3].

It applies much less to new work. Kempf chose Rust for Kyber and calls it excellent for new projects [3]. Van Rossum would still pick indentation in a green field [1]. He also tells young people to try unusual things: nine times out of ten you learn why everyone else does it differently, and one time out of ten you discover something better [1]. Lattner's advice runs the same way: "if everybody's going left sometimes it's pretty cool to go right." [2]

It also gives way when an outside constraint makes the old path unworkable. The royalty-free AV codecs exist, Kempf says, because HEVC licensing grew so costly that big platforms built their own codec [3]. Mojo exists because Modular needed a way to program a fast-multiplying range of AI hardware and no existing language fit [2]. In both cases the new thing was built for a reason other than taste [2][3].

None of these builders keeps everything forever. Python's standard library is doing spring cleaning of old modules [1]. VLC did end PowerPC support, at the personal cost Kempf describes [3]. Van Rossum expects Python itself to become, eventually, "a legacy language that plays an important role" underneath layers others build [1].

Failed breaks are not total losses either. Van Rossum says replaced technologies often introduced concepts that live on [1]. Ideas from Swift for TensorFlow, such as graph program abstraction, later influenced PyTorch 2 [2]. Libav's work ended up merged back into FFmpeg [3].

Two limits on the evidence should be stated. Mojo was about seven months old at the time, a 0.1 not meant for production, with classes, traits, keyword arguments and full module imports still to come, so these sources show the compatibility strategy as a design choice and not yet as a proven outcome [2]. And the sources contain no advocate for a Rust rewrite, no voice from Libav and no one from the Julia community, so the other side of each argument is reported only by its critics [2][3].

What To Do With It

1. Separate what you would choose in a green field from what you can change now. Van Rossum still prefers indentation and still says an established language cannot switch its block syntax [1].
2. Find the hidden layer before promising that nothing breaks. List the dependents that sit below the surface you are changing, as C extensions sit below Python and GCC's quirks sat inside ordinary headers [1][2].
3. Put the compatibility work on your side. Follow the Clang rule: implement the long tail yourself, since you will not persuade every user to rewrite [2].
4. Reuse what already exists before adding anything. Type hints shipped on syntax Python already had, with no changes to the language [1].
5. Offer the new behaviour as an option years ahead of making it the default. Van Rossum's sketch has an opt-in build several years before any switch, with a long warning for maintainers [1].
6. Give users a path that works one piece at a time. Mojo imports existing Python packages and lets code move across unchanged; Swift did the same alongside Objective-C [2].
7. Launch early, in the open, labelled as unfinished. Publish a roadmap and say not yet instead of promising everything, in the spirit of Lattner's "a hundred not yet for every yes" [2].
8. Run experiments outside the core. Let fast-moving tools try ideas before they become permanent syntax, as Python's type checkers do [1].
9. Hold a replacement to the old thing's tests. Admit a new language or a rewritten component only when it meets the same testing and stability bar, across the same matrix [3].
10. Use the new language for new projects, and read before you rewrite. That is Kempf's own practice with Rust at Kyber and his warning about existing code [3].

Sources

1. Guido van Rossum on Python 3.11's Speedup, Type Hints, the GIL and the Future of Programming · Lex Fridman Podcast
2. Chris Lattner on Mojo, Modular and the Future of Programming and AI · Lex Fridman Podcast
3. FFmpeg and VLC with Jean-Baptiste Kempf and Kieran Kunhya: Codecs, Assembly, and the Volunteers Behind Internet Video · Lex Fridman Podcast