Comment

Comment by superkuh

No... this headline is incorrect. You can't do that. I think they've confused the performance of running one of the small distills to existing smaller models. Two Arc cards cannot fit a 4 bit k-quant of a 671b model.

But a portable (no install) way to run llama.cpp on intel GPUs is really cool.

parent superkuh 1 year ago

Discussion

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton