Comment by mewpmewp2

I'm curious, what kind of setup is required to train something like that? Can it be done with 4090 and how long would it take? 50M or 25M parameters? And what is the most parameters 4090 can do?

This inspires few ideas.