Comment Comment by g42gregory This is why it’s so critical to have open source models.In a year or so, the open source models will become good enough (in both quality and speed) to run locally.Arguably, OpenAI OSS 120B is already good enough, in both quality and speed, to run on Mac Studio.Then $10k, amortized over 3 years, will be enough to run code LLMs 24/7.I hope that’s the future. parent g42gregory 1 year ago
In a year or so, the open source models will become good enough (in both quality and speed) to run locally.
Arguably, OpenAI OSS 120B is already good enough, in both quality and speed, to run on Mac Studio.
Then $10k, amortized over 3 years, will be enough to run code LLMs 24/7.
I hope that’s the future.