ilyakaminsky
Joined 2 years ago
Karma
40
Submissions
16
Member since
Dec 20
Recent submissions
Comment · 11 months ago
Comment · 1 year ago
How can I submit my service to your website? Is there a simpler way than creating a PR here? https://github.com/Klavis-AI/klavis/tree/main/mcp_servers
Comment · 1 year ago
Shameless plug -- check out speechischeap.com
I spent three months perfecting the speaker diarization pipeline and I think you'll be quite pleased with the results.
Comment · 1 year ago
> i can run it on consumer hardware for vastly cheaper than the cloud
Woah, that's really cool, CJ! I've been toying the with idea of standing up a cluster of older iPhones to run Apple's Speech framework. [1] The inspiration came from this blog post [2] where the author is using it for OCR. A couple of things are holding me back: (1) the OSS models are better according to the current benchmarks and (2) I have customers all over the world, so that geographical load-balancing is a real factor. With that said, I'll definitely spend some time checking out your work. Thanks for sharing!
[1] https://developer.apple.com/documentation/speech
[2] https://terminalbytes.com/iphone-8-solar-powered-vision-ocr-...
Comment · 1 year ago
Hmm… That's a good point. I recall a few instances where I went too far to the detriment of production. Having a trusty testing and benchmarking suite thankfully helped with keeping things more stable. As a solo developer, I really enjoy the development process, so while that bit is costly, I didn't really consider that until you mentioned it.
Comment · 1 year ago
TIL, thanks! I asked Claude to generate a simulator [1] based on your comment. I think it came out well.
[1] https://claude.ai/public/artifacts/1b921a50-897e-4d9e-8cfa-0...
Comment · 1 year ago
Fast is also cheap. Especially in the world of cloud computing where you pay by the second. The only way I could create a profitable transcription service [1] that undercuts the rest was by optimizing every little thing along the way. For instance, just yesterday I learned that the image size I've put together is 2.5× smaller than the next open source variant. That means faster cold boots, which reduces the cost (and providers a better service).
Comment · 1 year ago
Thanks for noticing. It took a lot of effort to optimize the pipeline every step of the way. VAD, inference server, hardware optimization, etc. But nothing that would compromise on quality. The audio is currently transcribed in its original speed. I'll be sure to publish something if I manage to speed it up without incurring any losses to the WER.
Comment · 1 year ago
I wouldn't describe it as "unusable" so much as needing to understand its constraints and how to work around them. I built a business on top of Whisper [1] and one of the early key insights was to implement a good voice activity detection (VAD) model in order to reduce Whisper's hallucinations on silence.
Comment · 1 year ago
Not yet. The gains in efficiency come from optimizing the speedup factor. Real-time audio cannot be processed any faster than 1× by definition.
Comment · 1 year ago
It's sustainable, but not enough to retire on at this point.
> Just wondering if I cam build a retirement out of APIs :)
I think it's possible, but you need to find a way to add value beyond the commodity itself (e.g., audio classification and speaker diarization in my case).
Comment · 1 year ago
I've already done that [1]. A fraction of the price, 24-hour limit per file, and speedup tricks like the OP's are welcome. :)
Comment · 1 year ago
This looks promising! Are you considering adding GPU support? It would be great if I could spin up a CUDA-enabled container right on Cloudflare, as opposed to using a Cloudflare worker to spin up a serverless GPU container on some other cloud provider.
Comment · 1 year ago
I've yet to try jj, but in git, my flow is to start a new feature with a WIP commit and then to `--amend` it with every change. I usually have a running TODO list in the commit message body that I check off along the way.
[1] https://github.com/alingse/ai-cli-log