Comment by wood_spirit

I like the view into the personal journey learning new things and sharing it.

Suggested next steps would be trying out DuckDB instead. My guess is it would smoke spark etc at this scale. Always good to learn different corners of the data landscape.