llama.cpp has great support for 65B, and I've been using it on a Linux box (I upgraded my RAM for that purpose). 64GB of RAM for a desktop is like $160 today, so it's not out of reach for most people here if they cared to.
Admittedly, it's quite slow and therefore not useful for chatting or real-time applications, and it's unreliable enough in its quality that I'd like to be able to iterate faster. Definitely more of a toy at this point, at least when run on CPU.
Admittedly, it's quite slow and therefore not useful for chatting or real-time applications, and it's unreliable enough in its quality that I'd like to be able to iterate faster. Definitely more of a toy at this point, at least when run on CPU.