Huh. This reminds me of the asymptotic equipartition theorem. Samples taken from higher and higher dimensional spaces will concentrate into a typical set.
Does model performance also concentrate into a 'typical case' where things work pretty well and a non-typical case where it's completely unpredictable as the number of parameters increase?
Does model performance also concentrate into a 'typical case' where things work pretty well and a non-typical case where it's completely unpredictable as the number of parameters increase?