Ffmpeg has an exceptionally terrible track record when it comes to security. People have been throwing fuzzers at it for as long as I remember and coming back with a nearly inexhaustible supply of memory corruption bugs. Here's an effort by one Googler a decade ago:
So, while it's a demo of the capabilities of LLMs, this should not be at all surprising. Ffmpeg is absolutely not something you should be running outside of a sandbox if you're touching any untrusted or user-supplied content. I know that people do, and these people are taking unreasonable risks.
https://security.googleblog.com/2014/01/ffmpeg-and-thousand-...
So, while it's a demo of the capabilities of LLMs, this should not be at all surprising. Ffmpeg is absolutely not something you should be running outside of a sandbox if you're touching any untrusted or user-supplied content. I know that people do, and these people are taking unreasonable risks.