pncnmnp
Joined 9 years ago
Joined 9 years ago
For example, I recently got the entire bundle for HBO's Rome for $9.99. That is ~50 minutes per episode (22 episodes), or $0.54/hour, which is not a bad deal! For a casual viewer like me, who might spend <6 hours each week watching TV, this should last a little more than 3 weeks. Recently, The Sopranos went on sale for $29.99, or $0.38/hour.
For streaming vs. buying to work, you just have to be patient about nabbing ~1-2 purchases per month and not be tempted to buy every single day.
The best part is the reduced cognitive load - you don't have to spend time browsing for content every day, and there are no ads, too.
I had some fun in my grad days building a private information retrieval (PIR) scheme so the shim server can do its job without knowing which link you clicked: https://github.com/pncnmnp/shimmey
https://historyofinformation.com/detail.php?id=525
> In a one page letter published in the London journal Nature on October 28, 1880, Henry Faulds was the first to propose the use of fingerprints as a system of identification, including the scientific identification of criminals.
From the paper itself:
> When bloody finger-marks or impressions on clay, glass, etc., exist, they may lead to the scientific identification of criminals. Already I have had experience in two such cases, and found useful evidence from these marks. In one case greasy finger-marks revealed who had been drinking some rectified spirit. The pattern was unique, and fortunately I had previously obtained a copy of it. They agreed with microscopic fidelity. In another case sooty finger-marks of a person climbing a white wall were of great use as negative evidence. .....
I've found that LiteRT-LM has a much lower DRAM footprint than Ollama. I've also made tons of optimizations in the code - for eg, you can do quite a bit with a 16k context window for a voice assistant while managing a good footprint, so I keep track of the token usage and then perform an auto-compaction after a while. I use sub-agents and only do deep-think calls with them, so the context window is separated out. In a multi-turn conversation, if Gemma 4 directly processes audio input, the KV cache fills up within a few turns, so I channel it all via Whisper.
Also, by far the biggest optimization is: 3-stage producer-consumer architecture. The LiteRT-LM streams tokens and I split them into sentences. A synthesizer thread then converts each sentence to audio via Kokoro TTS - the main thread then plays audio chunks sequentially. There's a parallel barge-in monitor thread. https://github.com/pncnmnp/strawberry/blob/main/main.py#L446
I did not want to use openWakeWord or Picovoice because they had limitations on which wake word you could choose. Alternative was to train a model of my own. So I created my own wake word detection pipeline using Whisper Tiny - works surprisingly well: https://github.com/pncnmnp/strawberry/blob/main/main.py#L143...
Also, I have VAD going with smart turn v3 (like I mentioned above) + I use browser/websocket for AEC + Barge-in (https://github.com/pncnmnp/strawberry/blob/main/audio_ws.py).
I'm using the MacBook's built-in microphones for this, though, and I haven't fully tested it with other microphones. I've been ironing out the rough edges on a daily basis. I should write a quick blog on this too.
Pipecat's smart turn model is really good for VAD - https://huggingface.co/pipecat-ai/smart-turn-v3
There is an interesting NSDI talk on the paper too - https://www.youtube.com/watch?v=kDJHA7TNtDk (2023)