I have been playing with Whisper speech-to-text recently (both CLI, and also . Depending on the model & computer performance, it seems like it can stream (listen and transcribe to text) audio. Maybe you can pipe that to a text-to-speech service (remote or local), and configure that as an audio input to OBS.
Are you live-streaming, or just recording video file to edit/publish later? If it's not live, that may relax some of the latency requirements.
Somewhat related, Descript mentions it does dubbing. I recently used this tool to edit a video, and I was impressed by how easy it was, and accurate too. It may not fit your realtime use-case, but might be worth exploring...
I have been playing with Whisper speech-to-text recently (both CLI, and also . Depending on the model & computer performance, it seems like it can stream (listen and transcribe to text) audio. Maybe you can pipe that to a text-to-speech service (remote or local), and configure that as an audio input to OBS.
Are you live-streaming, or just recording video file to edit/publish later? If it's not live, that may relax some of the latency requirements.
Related links:
- https://github.com/openai/whisper (speech-to-text for terminal)
- https://buzzcaptions.com/ (graphical interface to Whisper)
Somewhat related, Descript mentions it does dubbing. I recently used this tool to edit a video, and I was impressed by how easy it was, and accurate too. It may not fit your realtime use-case, but might be worth exploring...
https://descript.com/
Good luck!