joerawr

I think SSDs are the answer here. You'll get more space out of a several generation old SSD, and you still stuck to the PCI speeds so performance isn't going to be much different vs a card with DDR2 or 3 on it. (performance estimates are guess-timization, not calculations)

I really appreciate the focus on intelligence WITH token efficiency. I'd like to see that become the trend. Smartest per token metrics. Least tokens to accomplish the task above a certain success level. Most of my tasks would benefit from efficiency / token, but switching models constantly, and trying to guess the right model and effort level takes up too much of my processing.

Please do!

But it's about the journey not the destination. My current running local LLMs train of thought...