Comment Comment by alexkcd Don't know of a good single resource that is comprehensive. A high level overview of autodiff approaches can be found here: https://www.robots.ox.ac.uk/~tvg/publications/talks/autodiff...Some example implementations based on dual numbers: 1. http://www.ceres-solver.org/automatic_derivatives.html 2. http://www.imm.dtu.dk/~kajm/FADBAD/Newer ML frameworks do source to source transformations, which allows calculating the derivative without changing the function signature, but the concepts used remain the same. parent alexkcd 6 years ago
Some example implementations based on dual numbers: 1. http://www.ceres-solver.org/automatic_derivatives.html 2. http://www.imm.dtu.dk/~kajm/FADBAD/
Newer ML frameworks do source to source transformations, which allows calculating the derivative without changing the function signature, but the concepts used remain the same.