Comment by jacquesm

It is perfectly possible to build neural networks with other functions besides gradient descent.