znagengast

Takes me back immediately, really cool!

Maintainer of WhisperKit here, confirming we do exactly that for longform. We search for the longest "low energy" silence in the second half of the audio window and set the chunking point to the middle of that silence. It uses a version of the webrtc vad algorithm, and significantly speeds up longform because we can run a large amount of concurrent inference requests through CoreML's async prediction api. Whisper is also pretty smart with silent portions since the encoder will tell it if there are any words at all in the chunk, and simply stop predicting tokens after the prefill step - although you could save the ~100ms encoder run entirely with a good vad model, which our recently opensourced pyannote CoreML pipeline can do.

Turns out third party actually gets priority for ANE

I have to play devils advocate here because for every one of these cases, there is probably a dozen of similar stories where the ambitious new guy actually did nuke the system with a risky friday release and then logged off for the weekend xD

The thumbs up icon killed me

It's also largely dependent on the embeddings model as others have mentioned. Even if wikipedia doesn't have any words specifically referring to that monkey as "weird", the model itself would know to correlate this monkey's embeddings with the "weird" concept. The main issue with this particular implementation is the model used (all-minilm-l6-v2) which is designed for speed and efficiency over accuracy.

Isn't it crazy that the entirety of human knowledge can be condensed down to fit on an SD card.

Compared to how much progress the iPhone made from initial launch to now, the potential for this product line is very exciting.

What if you split up the training down to the literal vector math, and treated every macbook like a thread in a gpu, with just one big computer acting as the orchestrator?