Actually its one of those problems where it is worth not skipping. I have seen people applying neural networks to transform embeddings before doing a final dot-product where it could be proven (and indeed was confirmed in practice) that all of that was equivalent to a direct embedding.
Furthermore, SVD type of algorithm are orders of magnitude faster than NN to train (which is why they become used to train word embeddings, see Glove).
Furthermore, SVD type of algorithm are orders of magnitude faster than NN to train (which is why they become used to train word embeddings, see Glove).