Test obsessively: asserts, fuzzing and verification from FFmpeg to Cursor
September 29, 2026 · 25 min read
FFmpeg and VLC maintainers describe obsessive testing and reverse engineering, ThePrimeagen swears by asserts past the 5,000-line mark, and the Cursor team explores AI bug finding and the limits of formal verification. Together they offer a practical case for building correctness checks into code from the start.
The rules, numbers and steps worth keeping in front of you.
- Kieran Kunhya: bit-exactness means every codec implementation must produce exactly the same bits, so correctness becomes a mechanical check.
- FFmpeg's FATE: an automated test matrix across operating systems, compilers and CPU architectures.
- Kunhya: collect real-world sample files; a large varied set is what makes an exactness check meaningful.
- Kunhya and Jean-Baptiste Kempf: a safer language like Rust adds a layer but does not replace testing.
- ThePrimeagen: design choices start to bite at about 5,000 to 10,000 lines.
- TigerBeetle's rule: at least two asserts per function, kept on in production.
- Deterministic simulation testing runs the equivalent of hundreds of years of random input.
- Lex Fridman: treat warnings as errors and put asserts everywhere.
- ThePrimeagen: understand why old code exists before deleting it.
- Cursor team: frontier models are strikingly bad at finding bugs when simply asked.
- Arvid Lunnemark: comment dangerous code on every line; it helps humans and models.
- Cursor's shadow workspace: test AI changes in isolation before they touch your files.
- Rank bug reports by real-world severity, and send fixes with them.
- Formal proofs fit where intent is crisp; keep tests where intent is fuzzy or outside services are involved.
This is the short form. The full insight has every expert's reasoning, their own words and the sources.