The bigger issue is lack of generating novel content rather than a total "unravel". We focus on OOD images because our motivation is generating diverse environments, but these are much harder to play for longer vs images closer to the training videos. It is interesting because one of the things you gain when going from 1B->10B is the OOD images working at all. Note it is not even trivial to detect the character given our model does not train with any labels or have any inductive biases to do so.
Point of clarification -- we don't expect bigger models to be the only way to improve this and are working on innovations on the modeling side, however we don't want to overlook the significance of scaling either :)
Point of clarification -- we don't expect bigger models to be the only way to improve this and are working on innovations on the modeling side, however we don't want to overlook the significance of scaling either :)