Comment by deong

Even local optima are often pretty good in practice.

Also, it's a very good practice to iterate through the neighbors in a random order (if you're doing next descent, for steepest descent obviously it doesn't matter).