... and we know that if we really want to nail it we'd better just pay someone else to create 1,000,000 more harder problems for training (without looking at any in test set, of course). i.e. make the training set distribution similar to test set again.
Because the thing we have now is data-hungry. Your brain is pre-trained on other similar challenges as well. What's the point of requiring it to "generalize beyond the training distribution" with so few samples?
Really, I thought LLMs ended this "can we pretrain on in-house prepared private data for ILSVRC" flame war already.
Because the thing we have now is data-hungry. Your brain is pre-trained on other similar challenges as well. What's the point of requiring it to "generalize beyond the training distribution" with so few samples?
Really, I thought LLMs ended this "can we pretrain on in-house prepared private data for ILSVRC" flame war already.