Comment by thewisenerd

we get these hundreds of thousands of lines cost exports on a daily basis from our cloud provider

we process it 'daily' convert it from csv to parquet with a little post-processing for attribution, and have a small duckdb-powered dashboard/wrapper for reporting and dynamic queries