Comment

Comment by idonotknowwhy

If it's a Nvidia card 3000 series or newer, I'd try 4.0bpw ExllamaV3 if you haven't already. Otherwise it look like UD3.0 Q3_K_XL based on the Unsloth blog post.
parent idonotknowwhy 2 months ago

Discussion

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton