duffyjp
Joined 5 years ago
Joined 5 years ago
I'm shoehorning it back in the Optiplex that donated the ram, so it's not ready to go at the moment, but when I had it running on top of the motherboard box as a test I ran the (9B?) gemma4:e4b-it-q4_K_M since it can fit entirely in the 11gb vram. It flew, more than 50tk/s. A model that small isn't useful for coding, but there could be uses. I'd love to figure out a Wake-on-Use and use it as my personal ChatGPT. I'm not sure how that would work... Maybe proxy the LLM thru a Pi with a script to Wake-on-LAN the PC? It'll be a fun weekend project someday.
My always-on LLM is the dense Gemma4:31b that's not quite half in GPU on a 12gb 2060. It's really slow, but the quality is great and my use case is an automated queue so I'm not sitting there watching the output. I have another 2060 but unfortunately the PC won't POST with both installed for some reason.
I just picked up the DDR3 board, an Aliexpress "XD3" so I could reuse some DDR3 ram on a better CPU. Quad channel 1866MT/s is not bad!
I assumed it was a meta-data bug and posted an issue, but apparently nvfp4 doesn't necessarily mean nvidia-fp4.
My next GPU will be a Radeon, but they don't have anything to offer that can beat my 4080 from 2022...
I have a second used Radeon on the way for the other system. It's insane, but this is the only solution. If my monitor turns off or I switch inputs, I can't get it back when connected to the nvidia cards. I've done every troubleshooting step I can find, it's just a broken ecosystem.
It's how I play Minecraft with my kids when they get the itch. Sometimes if I know I'm only gonna be zoning out on Youtube at night I'll use to to save a few watts too.
It can do 1440p at 120hz, all on a really affordable phone. It's nice.
I have a free m.2 slot and may get a cheap optane drive and use that as SWAP. I've read they're not as fast as marketed for this, but at least I wouldn't have to worry about about wearing out my SSD.
I still have my install CD, though it has suffered from bit-rot and can't be read properly. :(
I gave one of the GPUs to my kid to play games on.
Buying more ram is no longer an option, so I added a 128gb swap partition on nvme. I incorrectly assumed with 64gb I didn't even need swap. No crashes since.
If you don't want to move partitions around, you can add a swap file. ChatGPT or whatever can give instructions.
I've declined the refresh I've overdue for. My 2021 model MBP has 32gb and a 1TB SSD. They're currently giving out the base model Air: 16gb and 256gb. No thanks.
We used to get whatever was most appropriate for our role, now we get whatever is cheapest.
When beef is expensive, you buy chicken. There's no alternative to RAM. If you need it, you've gotta pony up.
I did end up selling my best GPU, an RTX 4080. I had it for over three years and got ~80% of my money back.
You mentioned the COVID car market. The dealer we bought our Prius from sent us a letter during COVID offering to buy our car back sight-unseen for more than we originally paid. I regret not taking that deal, but at the time you couldn't replace it so what choice did I have...