Comment by slt2021

question about Arrow: the format seems to be not very space efficient.

I tried converting one of my parquet files from datalake from parquet to arrow and size difference is staggering. 20mb parquet -> 700mb arrow.

doesnt seem fit for datalake at all