Comment Comment by yieldcrv equilibrium in one or two more years on the consumer/prosumer sidethink Apple M6 or M7 with a currently unforeseen denser memory style, 256gb RAMa couple inference or cache improvements on the algorithmic side, using less ram for context windows and doubling token speed againdenser open source models, packing more experts for smaller active layersit'll still be expensive but like $8,000 - $13,000 instead of $450,000 worth of B200s parent yieldcrv 4 months ago
think Apple M6 or M7 with a currently unforeseen denser memory style, 256gb RAM
a couple inference or cache improvements on the algorithmic side, using less ram for context windows and doubling token speed again
denser open source models, packing more experts for smaller active layers
it'll still be expensive but like $8,000 - $13,000 instead of $450,000 worth of B200s