Comment Comment by legel You are wrong.Text tokens are high-dimensional vectors, not 8 bits per character. Every token has a deep embedding, e.g. 1024 float values per text token.DeepSeek-OCR proved 10x+ compression from visual embedding of text, which was a groundbreaking result. [1]Very cool to see OP's project hacking on this principle. It's still not lossless, as noted in the github, but is a promising research direction.[1] https://github.com/deepseek-ai/DeepSeek-OCR/blob/main/DeepSe... parent legel 4 months ago
Text tokens are high-dimensional vectors, not 8 bits per character. Every token has a deep embedding, e.g. 1024 float values per text token.
DeepSeek-OCR proved 10x+ compression from visual embedding of text, which was a groundbreaking result. [1]
Very cool to see OP's project hacking on this principle. It's still not lossless, as noted in the github, but is a promising research direction.
[1] https://github.com/deepseek-ai/DeepSeek-OCR/blob/main/DeepSe...