Lex Fridman Podcast

#496 – FFmpeg: The Incredible Technology Behind Video on the Internet

with Jean-Baptiste Kempf and Kieran Kunhya
7 May 2026 28 min read 2h 20m
TL;DR

FFmpeg and VLC are volunteer-built open-source projects that form the backbone of all video on the internet, yet most people don't know they exist. Video compression requires balancing human perception with mathematical complexity—codecs achieve 100-1000x compression by exploiting spatial and temporal redundancy while matching how human eyes and ears actually work. The real genius is not the final codec, but the philosophical commitment to handling broken files, supporting any format, and maintaining zero tolerance for corporate backdoors.

Key Moments
Jean-Baptiste Kempf
“The important is, is your code good? We care about excellent code. We don't care who you are. Like maybe you're a dog. I don't care, right? I don't care where you come from. I need to look at your code.”
Kempf describes the meritocratic philosophy that defines the VideoLAN community
Kieran Kunhya
“FFmpeg is probably one of the biggest CPU users in the world. Everything we've just said in the past couple of minutes, every sentence is someone's lifetime's work. There are books about every sentence. So the level of complexity in many cases is inordinate.”
Kunhya emphasizes the staggering depth and breadth of knowledge embedded in FFmpeg
▶ 0:31
Kieran Kunhya
“The intelligence agencies tried to, like, say, 'Can you put a backdoor in VLC?' Yes. Two of them. Well, what did you say? No. Well, I was a lot less polite.”
Kempf reveals that government agencies have attempted to compromise VLC's security
▶ 1:43
Jean-Baptiste Kempf
“When you're playing video which are on UDP, right, in network, they might be damaged, right? So you don't trust your inputs, and this is very important into the security is that you don't trust your inputs. So everything in VLC is prepared to work with broken files.”
Kempf explains VLC's core philosophy of robustness, born from streaming video over unreliable networks
▶ 23:50
Jean-Baptiste Kempf
“Compression is not like a ZIP, right? A ZIP, you have data in, you get data out, right? And you try with all the ZIP compression to arrive with the limit. Here we are degrading the signal, right? And so we need to degrade both the audio and the video signal in the best way possible.”
Kempf distinguishes lossy video compression from lossless data compression, explaining why human perception is central to codec design
▶ 16:04
About the Guests
›
Jean-Baptiste Kempf is president of VideoLAN and a key figure behind VLC, the legendary open-source media player downloaded over 6 billion times. Kieran Kunhya is a longtime codec engineer and FFmpeg contributor, known for the viral FFmpeg Twitter/X account that celebrates low-level software engineering. Together they represent the volunteer engineering community behind FFmpeg and VLC—the invisible infrastructure powering video across YouTube, Netflix, Chrome, and virtually every platform on the internet.
Takeaways
1
Video codecs exploit human perception limits Modern video compression doesn't just remove redundancy—it strategically degrades signals in ways humans cannot perceive. Converting RGB to YUV and scaling chroma separately can halve file size without visible quality loss. Each codec generation achieves 30% better compression but requires orders of magnitude more CPU power to encode, reflecting a deliberate trade-off between compute cost and compression efficiency.
2
Robustness through defensive architecture VLC's legendary reliability stems from its origins in UDP-based streaming where packets get lost. The entire codebase is designed to recover from broken files, incorrect metadata, and malformed streams—treating every input as potentially corrupted. This philosophy, baked in from day one, is why VLC can play formats that other players reject, and why it became the de facto recovery tool for damaged media.
3
Open-source infrastructure powers trillion-dollar platforms FFmpeg and VLC, built entirely by volunteers, form the backbone of video delivery for Netflix, YouTube, Chrome, Discord, and most internet platforms. A grandmother's home videos and corporate services use identical technology stacks. This represents a remarkable feat of invisible engineering: billions of people consume video daily without knowing the software that makes it possible is maintained by a small community that refuses corporate compromises or backdoors.