mrajcok
Joined 12 years ago
Karma
63
Submissions
30
Member since
Feb 23
matthewrajcok [ AT ] gmail [ DOT ] com
View on Hacker NewsJoined 12 years ago
Will have Llama 4 Maverick running in 4bit quantization (typically results in only minor quality degradation) once llama.cpp support is merged.
Total hardware cost well under $50,000.
The 2T Behemoth model is tougher, but enough Blackwell 6000 Pro cards (16) should be able to run it for under $200k.
When the driver is not on a trip, we turn on both apps. When they are on a trip, we turn off the app not in use.
We've thought about pitching in the car but the cost doesn't work out.