The Allen Institute (a non-profit) just released the Molmo 2 and Olmo 3 models. They trained these from scratch using public datasets, and they are performance-competitive with Gemini in several benchmarks [0] [1].
AMD was also able to successfully train an older version of OLMo on their hardware using the published code, data, and recipe [2].
If a non-profit and a chip vendor (training for marketing purposes) can do this, it clearly doesn't require "burning 10 years of cash flow" or a Google-scale TPU farm.
The Allen Institute (a non-profit) just released the Molmo 2 and Olmo 3 models. They trained these from scratch using public datasets, and they are performance-competitive with Gemini in several benchmarks [0] [1].
AMD was also able to successfully train an older version of OLMo on their hardware using the published code, data, and recipe [2].
If a non-profit and a chip vendor (training for marketing purposes) can do this, it clearly doesn't require "burning 10 years of cash flow" or a Google-scale TPU farm.
[0]: https://allenai.org/blog/molmo2
[1]: https://allenai.org/blog/olmo3
[2]: https://huggingface.co/amd/AMD-OLMo