Joined 11 months ago
Banafo is a small company that trained Kroko, a streaming Zipformer2 transducer, and released community models on Hugging Face: https://huggingface.co/Banafo/Kroko-ASR
I picked a transducer because it suits a microcontroller. The encoder is 98% of the weights, and it runs once per 1.28 s chunk of audio, so each weight is read from PSRAM once and reused for every frame of the chunk. The decoder is tiny (under 2% of the model) and only remembers the last two tokens. On these chips the limit is reading the weights from memory, not the math. A Whisper-style model is the opposite: most of its weights are in a decoder that runs once per token, rereading them every time.
The model is Kroko (by Banafo), a 65.7M-parameter streaming Zipformer2 transducer. As released it is 71 MB in int8, which does not fit the board's 32 MB of PSRAM, so I quantized the weights to 2 and 4 bits (26.65 MB, groups of 128 with a Hadamard rotation, the bit width chosen per matrix) and wrote my own engine: integer arithmetic throughout, with the hot loops hand-written in PIE assembly.
Numbers, measured on all of LibriSpeech:
test-clean 4.34% WER, test-other 10.26% (the original int8 model on a PC: 3.94% and 9.17%).
1.28 s of audio processed in 0.90 s (0.70x real time), both cores at 400 MHz.
A word is final about 2 s after it is spoken.
One thing I care about: the chip computes exactly the same bits as the engine's PC build, so every accuracy number above is the chip's, not a simulation. Getting there meant replacing libm (newlib and glibc disagree in the last bit of log/exp/cos) with deterministic math on both sides.
There are also hotwords (phrases to listen for, from a text file on the SD card) and six voice-control demos driven by simple word rules, no LLM.
Limits: English only; it needs the chip revision v3 (P4X) with 32 MB PSRAM; accuracy is measured on clean read speech, noisy rooms not yet. You can flash it from the browser; the weights are on Hugging Face.
Video: https://www.youtube.com/watch?v=oe-2MowdsSs
Happy to answer questions about the quantization, the kernels or the streaming.
