Comment

Comment by charcircuit

Of course. Models don't actually require VRAM. Nor do they require regular RAM. You could have 1 GB of RAM and swap the model to disk as you need different parts of it. And if you didn't have enough disks you could access weights via a network connection.
parent charcircuit 2 months ago

Discussion

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton