Comment by evilmathkid

train from scratch only during the 12 hours allowed on Kaggle

Other competitions have implemented things like this before. Eg: OpenAI's Parameter Golf and Keller Jordan's Modded NanoGPT Speedrun