Comment by x0x0

The Hessian is too expensive and too big.

lbfgs is quite common for eg regression w/o l1 penalties.

MPI is not great at even high hundreds of cores; it's too much work to build redundancy / retry / restart / clean failure in. You really need a framework that helps with this.