aria

I'll try! I've got a full time management job so this was a fun way to code again. The library is pretty small and I think easy to follow if you want to learn

Definitely doing the standard back-prop algorithm. Check out the code, it's pretty straightforward and the whole lib is about 2,500 lines

Author here, happy to answer any questions!

Author here. This is coming up. Will do OpenCL first and then CUDA.

Author here, happy to answer questions and hopefully convince someone to give the library a try.

The interesting part of BFGS is that it directly approximates H^{-1} rather than H.

Want to try a refresh, I think it ought to be fixed now. Thanks

In practice you only need to do $H^{-1} g$; L-BFGS stores the {s_k} and {y_k} vectors which allow you to do the $H^{-1} g$ directly rather than needing to ever form the hessian or its inverse. There are techniques that aren't BFGS-based which approximate the hessian rather than the inverse and in that case you'd be better off solving.

Typo is fixed, thanks!

I think that's explicitly mentioned in the Quasi-Newton section that you only need to implicitly multiply and not form the matrix.

What browser are you in? It just using MathJax and appears to work in WebKit and Firefox.

Author here, happy to answer any questions.

Question 1: What is the first bit in your unique 33-bit string? Question 2: What is the first bit in your unique 33-bit string? ...