rguiscard

Joined 7 years ago

Karma
3,885
Submissions
467
Member since
Mar 6
[ my public key: https://keybase.io/rguiscard; my proof: https://keybase.io/rguiscard/sigs/mVHEOkVvrO76COiMnXbWWNeb7m_dIYLKst5PcEKWXcM ]
View on Hacker News

Recent submissions

Comment · 1 month ago
I wonder the same thing for Bonsai 2. ByteShape offers 5 models from IQ2_XXS-2.56bpw (8.8GB), IQ3_XXS-2.88bpw (9.9GB), IQ3_XS-3.01bpw (10.4GB), IQ3_S-3.23bpw (11.0GB) to IQ4_XS-3.84bpw (13.1GB). Their benchmarks show gradual improvement with size and users can pick one to fit theirs need. Bonsai-2-27B now is about 8.6GB. It might be good to have a quaternary version around 10-11GB to fit a computer with 16-24GB RAM.

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton