I built audio-chat-box to give a program a voice without sending audio anywhere. It reads text aloud with Kokoro, transcribes with NVIDIA's Parakeet TDT 0.6B v2, and can listen hands-free: your program streams microphone samples, Silero VAD finds each thing said, and Parakeet transcribes it when the speaker pauses. It all runs locally through sherpa-onnx.
To try it without building anything, there's a small browser demo in Docker. You speak, it shows what it heard, and it reads it back in any of Kokoro's 54 voices:
docker run --rm --platform linux/amd64 -p 127.0.0.1:8080:8080 pnxstudios/audio-chat-box-demo
then open http://localhost:8080 (that address, not 127.0.0.1). The models are inside the image, so it needs no network, account or API key. It's amd64 only for now: on Apple Silicon it runs under emulation, about 4 s to speak a phrase, against under 1 s built natively.
It's a separate process, not a library. Your program starts it and exchanges JSON messages over stdin/stdout, one per line, so any language that can start a process can use it. It never touches the mic or speakers itself; your program records and plays. There's also a practical reason: sherpa-onnx's build includes eSpeak NG, which is GPL-3.0. As a separate program it's GPL, with the source of the native libraries it links published as a release, but your app starts it rather than linking it.
A few details:
- Hotwords: your program says which words to expect, and it favours them where the audio is ambiguous. It grew out of a chess app, so "knight" over "night".
- Speed: on an M1 Max, built natively, both models load in about a second, and a sentence is spoken in under one.
- Models: about 835 MB, downloaded once on first use and checked against SHA-256s built into the program. --offline never downloads.
- Protocol: fully documented in docs/protocol.md. The Docker demo's web server is itself a small client of it (demo-web/), if you want a worked example.
It began as the voice of Solrac (https://solrac-chess.com), an AI chess coach I'm building. Its listening was tuned on chess moves said at speed-chess pace. I took out everything chess-specific so any project can use it. I wrote it with Claude Code as a pair programmer; the commits say so.
Current limits: using it from your own program means building it with Rust (the Docker image is a demo), transcription is English only, and it handles one listen at a time. I'd love feedback, especially from anyone wiring voice into an agent or assistant.
To try it without building anything, there's a small browser demo in Docker. You speak, it shows what it heard, and it reads it back in any of Kokoro's 54 voices:
then open http://localhost:8080 (that address, not 127.0.0.1). The models are inside the image, so it needs no network, account or API key. It's amd64 only for now: on Apple Silicon it runs under emulation, about 4 s to speak a phrase, against under 1 s built natively.It's a separate process, not a library. Your program starts it and exchanges JSON messages over stdin/stdout, one per line, so any language that can start a process can use it. It never touches the mic or speakers itself; your program records and plays. There's also a practical reason: sherpa-onnx's build includes eSpeak NG, which is GPL-3.0. As a separate program it's GPL, with the source of the native libraries it links published as a release, but your app starts it rather than linking it.
A few details:
- Hotwords: your program says which words to expect, and it favours them where the audio is ambiguous. It grew out of a chess app, so "knight" over "night".
- Speed: on an M1 Max, built natively, both models load in about a second, and a sentence is spoken in under one.
- Models: about 835 MB, downloaded once on first use and checked against SHA-256s built into the program. --offline never downloads.
- Protocol: fully documented in docs/protocol.md. The Docker demo's web server is itself a small client of it (demo-web/), if you want a worked example.
It began as the voice of Solrac (https://solrac-chess.com), an AI chess coach I'm building. Its listening was tuned on chess moves said at speed-chess pace. I took out everything chess-specific so any project can use it. I wrote it with Claude Code as a pair programmer; the commits say so.
Current limits: using it from your own program means building it with Rust (the Docker image is a demo), transcription is English only, and it handles one listen at a time. I'd love feedback, especially from anyone wiring voice into an agent or assistant.