Comment Comment by lunixbochs I did a silly experiment to compress word embeddings with jpeg - to see how it collapses semantically as you decrease the quality.https://bochs.info/vec2jpg/This was a very basic experiment. I expect you could perform the DCT more intelligently on the vector dimensions instead of trying to pack the embeddings into pixels, and get higher quality semantic compression. parent lunixbochs 2 years ago
https://bochs.info/vec2jpg/
This was a very basic experiment. I expect you could perform the DCT more intelligently on the vector dimensions instead of trying to pack the embeddings into pixels, and get higher quality semantic compression.