Comment by lsorber

Have you benchmarked this against pickling those data files? In our experience, parquet's overhead isn't worth it for smaller data files.