I've seen the power of moving systems to more machine readable file formats such as Parquet, Arrow as opposed to storing them as csv, json etc. For people who are actually making these design choices in my experience it has always been a better idea to prefer smarter formats over readable ones for large scale systems. They really help in longer term for :
- cost (more maintainable from the tech overhead required).
- cost (more maintainable from the tech overhead required).
- storage (lower size)
- compute (faster reads and indexing)