I had a similar experience when i was in highschool and learning python. I wrote a image->ascii image convertor using PIL as you did, one image would take a few hundred ms and it was fine, but as i played with it and added more features/learned more i tried to do it on video frames extracted with ffmpeg and it would take atleast 4x the duration of the video or even more to convert it in ascii video.
Few months ago i coded it in go since i got back into programming and it was significantly faster(The overall code was also smaller but maybe that's just because i have a bit more experience now) even tho in both cases the implementations was the most naive one i could come up with, without any optimizations.
Edit:
I went ahead and tested both, took me 30 mintues to set them with identic parameters on 3602 frames(roughly 1 minute video)
Keep in mind that this is just a naive implementation with 2 nested loops itereting in a step of 12 over the pixels of the image, no fancy math or numpy used in the python version. Image for reference[0].
Also keep in mind that the step and length of the ascii charmap used affects performance but here i used a simple one "_^:-=fg#%@" so it's a bit easier for the python code, here are the results on my i7 9th gen laptop
1 Minute video 720p 60fps
GO - Execution time: 359647 ms, 5.9 minutes ("asciifying the frames") + 30~ seconds ffmpeg video conversion
Python 3.12.3 - Execution time: 0:16:25.002945, minus ~25-30 seconds ffmpeg
[0]: https://i.postimg.cc/CK30z1jt/frame-000039.png
[1]: Video using a more detailed ascii charmap(" .,:;i1tfgLCG08@#") in the go version -> https://streamable.com/7h8jc3
Edit: I went ahead and tested both, took me 30 mintues to set them with identic parameters on 3602 frames(roughly 1 minute video) Keep in mind that this is just a naive implementation with 2 nested loops itereting in a step of 12 over the pixels of the image, no fancy math or numpy used in the python version. Image for reference[0]. Also keep in mind that the step and length of the ascii charmap used affects performance but here i used a simple one "_^:-=fg#%@" so it's a bit easier for the python code, here are the results on my i7 9th gen laptop
1 Minute video 720p 60fps
GO - Execution time: 359647 ms, 5.9 minutes ("asciifying the frames") + 30~ seconds ffmpeg video conversion
Python 3.12.3 - Execution time: 0:16:25.002945, minus ~25-30 seconds ffmpeg
[0]: https://i.postimg.cc/CK30z1jt/frame-000039.png [1]: Video using a more detailed ascii charmap(" .,:;i1tfgLCG08@#") in the go version -> https://streamable.com/7h8jc3