Comment Comment by pizza234 > but this is how progress happens.This is progress in the same way that a man climbing a tree is making progress toward reaching the moon.This project is essentially the MoE-of-the day, with some platform-related optimizations.> Keep going! I personally can't wait for the day when a 1t param model runs off a $200 SSD instead of a $50k rack of Nvidia chips.That won't happen. Projects like this just give the illusion that that will be possible. parent pizza234 3 months ago
This is progress in the same way that a man climbing a tree is making progress toward reaching the moon.
This project is essentially the MoE-of-the day, with some platform-related optimizations.
> Keep going! I personally can't wait for the day when a 1t param model runs off a $200 SSD instead of a $50k rack of Nvidia chips.
That won't happen. Projects like this just give the illusion that that will be possible.