I agree with everything you said. I just mean that a single node may be slow when processing those parquet files in a complex aggregation, bottlenecked on network IO or CPU or available memory.
If the thesis here is that most datasets are small, fair enough - but then why use a lake instead of big postgres, yknow?
If the thesis here is that most datasets are small, fair enough - but then why use a lake instead of big postgres, yknow?