Comment by jdthedisciple

I might be wrong but mathematically we would call a 2-dimensional object a matrix and a 3- or higher-dimensional object a tensor.

Now I guess since the described auto grad algorithm works for arbitrarily high dimensional objects it makes sense to call these objects tensors.