Comment by jimfleming

I'd love for someone who knows more to chime in but its my understanding that gradient descent does not perform well on non-differentiable functions and only finds local optima.

Wikipedia seems to confirm this: http://en.wikipedia.org/wiki/Gradient_descent#Limitations