gauravm

New blog post on Efficient AI Techniques: KV Caching and KV Sharing.

Efficient training and inference is table stakes for LLMs these days, and these two algorithmic efficiency techniques work really well for reducing LLM latency as well as memory usage, while retaining the model performance. Feel free to give it a read, and drop a note if I missed something.

We have been working on a book that focuses on deep learning efficiency techniques such as quantization, pruning, distillation, etc. for both server-side as well as on-device (smartphones, IoT, etc.) applications. We had earlier released the first four chapters for anyone to read for free.

We now have a new chapter focusing on sparsity and clustering, two advanced compression techniques that you can use to reduce the footprint of your model (size, latency, etc.) while retaining your model accuracy. You can read the five chapters released so far, and go through the accompanying codelabs in the form of Jupyter notebooks.

We hope that our readers can make their models 4-20x smaller, faster, and better in quality. We would truly appreciate any sort of comments / feedback.

Book: efficientdlbook.com Feedback: [email protected]

There is Tensorflow Lite that Google just opensourced.

Sadly, there is no 'R' anymore :-/

I really like this approach of gamified interview prep, kind of like Duoling style learning. Practicing for interviews is hard, and this makes it a little more fun :) Good luck!

I wanted something easy to use to quickly get an idea of how much explicit content could we be dealing with. The main challenge was dealing with a multi-lingual database. I didn't even find a naive classifier.

Though I don't have time/RoI to improve this, but potential ideas are to use labeled data to cluster porny words and get a probablistic metric of porni-ness of a sentence.

Thanks. This quick idea worked for my cases, because there were few potential false positives. But your idea around using a regex style matcher should be good.