I'm not up-to-speed with the current state of sandboxing in browsers, but in principle it's (on modern operating systems) not especially hard for them to sandbox the decoding into a separate process with basically no privileges beyond rendering a video stream. It's a bit trickier if we're only considering demuxing and delegating decoding to the hardware, but that's a much smaller attack surface.
A manually run ffmpeg on the command line does nothing to restrict its privileges, and its security model has very little interest in doing so, while browsers very much have.
A manually run ffmpeg on the command line does nothing to restrict its privileges, and its security model has very little interest in doing so, while browsers very much have.