Comment

Comment by petrizhang

I ran into exactly this while profiling a Parquet-based vector index on DataFusion. The vectors all have a fixed dimension, but LIST decoding and rebuilding the Arrow arrays still showed up as a significant CPU cost. Have you looked at whether a similar fast path could be added to parquet-rs?
parent petrizhang 3 months ago

Discussion

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton