Insight | Speed is a feature: why top engineers obsess over latency
FFmpeg's handwritten assembly, Cursor's KV cache and speculative edits, and Perplexity's work on indexing and latency all show teams treating milliseconds as product quality. The pattern explains why performance work, often dismissed as premature optimization, is central to software people love.
People who build software that others love treat speed as part of the product, not a finishing touch. Three very different teams describe milliseconds and CPU cycles as a measure of quality, and they build the whole architecture around that from the start.
The volunteers behind FFmpeg and VLC give the most extreme version. Jean-Baptiste Kempf and Kieran Kunhya describe handwritten assembly that typically runs 10 to 50 times faster than C. Their AV1 video decoder holds roughly 30,000 lines of C and 240,000 lines of handwritten assembly, built under the motto "every cycle matters." They reject the claim that compilers can match this, saying the gap is multiples, not a few percent. Up to 45% of video files cannot be decoded on a graphics chip and fall back to the processor, which is why those cycles count. Speed is kept safe by exact testing: every decoder must produce identical output, checked across a huge matrix of platforms.
The founders of the Cursor code editor make the product argument. They say a code editor should be fun, that much of fun comes from speed, and that they throw out experiments that work but are not fun. Their autocomplete only helps if it appears instantly, so they reuse a cache across keystrokes, warm it while you are still typing, and compute the next suggestion in advance. They also invented speculative edits, which use the existing code as a draft so changes stream faster and people can review before the edit finishes. Without the cache, they say, they would destroy latency and overload their graphics processors.
Aravind Srinivas of Perplexity inherited the habit from Larry Page, who tested Chrome on old laptops. Srinivas tests his product on in-flight Wi-Fi, tracks the slowest 10% and 1% of requests at every component, and sweats details like how quickly the phone keyboard appears. He pairs this with another Page principle: "the user is never wrong."
They agree that speed is designed in, that small details are real quality, that speed and cost are the same problem, and that fast must also be correct. They differ on who does the optimizing. The FFmpeg engineers see it as a human craft that current AI models struggle with, while the Cursor team builds speed with machine learning and bets that AI will help. They also mean different things by speed, and all accept that some tasks are worth waiting for.
The article adds caveats. Speed never excuses wrong answers. Rewrites for their own sake are a trap, and not every layer needs custom work. None of the sources cites data on how latency affects user retention; the claim that speed drives love rests on the teams' own experience.
What to do with it: write speed into your design principles. Measure the slowest requests at every component, not averages. Test on bad connections, old hardware and against rivals. Sweat the small interactions. Stop recomputing by caching and precomputing. Put the cheap, fast component on the hot path. Guard speed with exact tests, and optimize for what users actually perceive.
