apaszke

PyTorch was never a Chainer fork. The whole codebase are C libs from Lua Torch, and a bunch of Python code that was written entirely for this project. Chainer was an inspiration, but no code was ever shared between those two projects.

PyTorch uses NVIDIA NCCL for multi-GPU communication (under BSD license). Gloo is only one of the three backends that can be used for distributed communication out of the box.

It's not controlled by Facebook in any way. It's true that a large part of the core team works there, but development is public and guided by community needs first.

It's not like you have to give up a lot - the graphs are simple data structures and creating them is not the expensive part of the training. The computation has to be re-done at every step in a static framework too, and this is the part that matters.

gloo is only one of the three currently supported backends. One can easily switch to MPI, and pick an implementation that comes with a license you want.

It supports numpy, and conversion from PyTorch tensors to numpy arrays is a matter of calling the numpy() method.

To not have the kernel launch overhead you'd need to stop launching GPU kernels but that's now how things work in any framework ;)

These ops are just not needed in PyTorch. while is just a Python while loop. Scan is a for loop, map is a list comprehension that applies modules. No need for anything fancy.

You save the parameters and the code of the model definition

torch?

No! Python 0 based indexing everywhere.