Comment by jeremytian

From my experience, models that drive optimizers in a loop often get stuck in local minima. ofc, this is just correlation, and not truly indicative, but if you look at the first graph, analyzing precision vs cost, all three model-driven optimizers just bounce back and forth between a precision range, not really expanding the horizon or getting too much worse than the orignal. from our analysis on this data, model-driven optimizers typically don't start from scratch again, they just continue down their optimization path.