antoine-levitt

Could you share a bit more details on your config?

You can easily test a gradient using only the forward pass by doing f(x+h) ~ f(x) + dot(g, h) for a random h

I just skimmed the article and didn't watch the video, but the bit about backpropagation is just wrong. Backpropagation doesn't compute an inverse of the jacobian, it computes its transpose. (although a similar idea to backpropagation could possibly be used to compute an inverse of several reversible layers, but that's not typically how neural networks work)

I'm willing to bet the actual reason the transition went so smoothly and the library doesn't pay that much in per-article fees is because everybody just used sci-hub.

> The name quantum in quantum theory is related to the fact that in a separable Hilbert space, any set of mutually orthonormal vectors is countable.

Pretty sure the name quantum comes from the fact that some physical phenomena (eg absorption spectrum) get discrete allowed values. That in principle has nothing to do with separability (eg you can come up with non separable spaces which have operators with discrete spectrum). In fact presenting things this way is pretty confusing since separable Hilbert spaces do support operators with continuous spectrum (which is not obvious!) As far as I know separability is mostly technical, and often added to make life a bit simpler, since it's pretty hard to come up with useful non separable Hilbert spaces.

(in mice)

... And so that constraint forces R1 to be zero? Note that finding a solution to the problem originally stated is equivalent to a linear programming problem, which (for such a simple problem) can easily be solved exactly.

Do not be fooled, julia is not a oo language, despite some similarities. If you try to replicate oo patterns in Julia it's not going to work well, but in idiomatic julia I've never found the need for concrete subtyping: generally julia under emphasizes class hierarchies in favor of types as dispatch vessels and data holders. It's very different from OOP but when you get used to it it's quite nice. The things I miss from oop is the consistency (when there's just one way to model the world you don't have to think quite as much about design) and things like tab completion for method discovery.

MPI.jl does not have an array wrapper type, it just exposes the bare mpi calls. Some other packages do use it to build array types.

Are you sure that the difference is due to the allocations? I would expect this to be dominated by matrix multiplies or svds. Are you comparing this with the same blas/LAPACK?

Edit : OK, I see those are small matrices. Then Staticarrays should be a nice contender here, both for speed and readability.