pythongiant

Joined 7 years ago

Karma

14

Submissions

26

Member since

Feb 26

@pythongiant(github) is an AI/ML researcher and engineer, currently a Researcher at Microsoft Research.

View on Hacker News

Recent submissions

Bodhan.ai releases 1.2B ASR model that covers 22 Indian Languages bodhan.ai

2 pythongiant 1 month ago 0 comments

Comment · 2 months ago

Tenosra is all about making AI more efficient and resourceful you can read more abour it at: https://www.tenosra.com/proxima

Show HN: Proxima serves 4x more requests with no hardware change on vLLM github.com

3 pythongiant 2 months ago 1 comments

Comment · 3 months ago

Hey guys, i'm especially interested in feedback on the kernel design and integration onto mlx lm.

Its on pypi as well as a simple pip install mlx-turboquant :P

Show HN: TurboQuant for mlx-lm (Apple Silicon) github.com

1 pythongiant 3 months ago 1 comments

Comment · 5 months ago

KVBoost is a drop-in replacement for AutoModelForCausalLM. Same API surface (KVBoost.from_pretrained(...), engine.generate(...)), but with cross-request KV reuse, FlashAttention-2, AWQ layer streaming, and speculative decoding bolted on.

Comment · 5 months ago

Here's the repository incase anyone wants to have a look at the code. leave a star if you find it interesting :P https://github.com/pythongiant/KVBoost

Comment · 5 months ago

Thanks! this is a weekend project that i am working on in the side just to learn more about ml engineering and custom cuda kernels. didnt think much about the website