Comment

Comment by azimafroozeh

Arrow is primarily an in-memory format, while Parquet is commonly used for on-disk storage. Typically, data is stored in Parquet and then read into memory as Arrow. FastLanes is a new on-disk file format, comparable to Parquet, but offers around 40% better compression and faster decoding thanks to its data-parallel encoding design.

Disclaimer: I'm the first author of the paper.

Discussion

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton