Speed is a feature: why top engineers obsess over latency
September 29, 2026 · 23 min read
FFmpeg's handwritten assembly, Cursor's KV cache and speculative edits, and Perplexity's work on indexing and latency all show teams treating milliseconds as product quality. The pattern explains why performance work, often dismissed as premature optimization, is central to software people love.
The rules, numbers and steps worth keeping in front of you.
- Core idea: speed is designed in from the start, not bolted on at the end.
- FFmpeg: handwritten assembly typically runs 10 to 50 times faster than C.
- VideoLAN's AV1 decoder: about 30,000 lines of C and 240,000 lines of assembly.
- Kieran Kunhya and Jean-Baptiste Kempf: compilers do not close the gap; it is multiples, not a few percent.
- Kunhya: up to 45% of files need software decoding on the processor.
- FFmpeg guards speed with exact tests: every decoder must produce identical output across platforms.
- x264's lesson: optimize what human eyes perceive, not the easy mathematical metric.
- Cursor team: much of fun comes from speed; cut experiments that work but are not fun.
- Cursor: reuse the cache across keystrokes, warm it while the user types, precompute the next suggestion.
- Cursor: let the expensive model sketch and a small trained model apply the change.
- Cursor: more context makes requests slower and more expensive, so keep the bar for relevance high.
- Aravind Srinivas: track P90 and P99 latency at every component, plus time to first token.
- Srinivas: test on in-flight Wi-Fi and benchmark against rivals; sweat cursor and keyboard timing.
- Caveat: no source cites data linking latency to user retention.
This is the short form. The full insight has every expert's reasoning, their own words and the sources.