Comment

Comment by ilc

Any one weight, but all of them. And also crushing the architecture itself?

I wouldn't pick up 400gb of hardware to run in that mode. I might try it for fun, but even then you are looking at handling a 95GB active parameter set.

This is NOT a model for most home labs. I'm sure some can and will use it. But most, should steer clear.

parent ilc 2 months ago

Discussion

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton