I really don't feel that there is magic in PyTorch or jax, but that may be because I have written my own autograd libs.
In PyTorch you have a graph that is created on runtime by connecting the operations together in a transparent manner.
Jax may feel a bit magic, but all that's done is sending / splitting tracers and recording the operations and compiling; by limiting the language, you have controlled branching with the proper semantics.
---
The main reason I have had such with Julia is simply because of how early / soon I ended up needing to use them whereas with the other languages, you can get away without getting into the very messed up things.
In PyTorch you have a graph that is created on runtime by connecting the operations together in a transparent manner.
Jax may feel a bit magic, but all that's done is sending / splitting tracers and recording the operations and compiling; by limiting the language, you have controlled branching with the proper semantics.
---
The main reason I have had such with Julia is simply because of how early / soon I ended up needing to use them whereas with the other languages, you can get away without getting into the very messed up things.