> You can push this further by combining quantization with truncation or PCA. The resulting vectors can be dramatically smaller while still preserving a surprising amount of retrieval quality.
Counterintuitively - quantisation can also be combined with a random rotation step before the quantisation. A random rotation spreads information across more dimensions, allowing more aggressive quantisation without losing accuracy. Ironically - almost the opposite of a PCA.
I do wonder if relevant here though. It relies on the embeddings having "structure", i.e. that principal components point along basis vectors, which may not be the case with text embeddings.
> You can push this further by combining quantization with truncation or PCA. The resulting vectors can be dramatically smaller while still preserving a surprising amount of retrieval quality.
Counterintuitively - quantisation can also be combined with a random rotation step before the quantisation. A random rotation spreads information across more dimensions, allowing more aggressive quantisation without losing accuracy. Ironically - almost the opposite of a PCA.
I do wonder if relevant here though. It relies on the embeddings having "structure", i.e. that principal components point along basis vectors, which may not be the case with text embeddings.
Source: https://research.google/blog/turboquant-redefining-ai-effici...