Comment by jonathanlb

I ran a similar personal project where weekly I scraped a country's news stories across curated sources using Brave News API and used an LLM to dedupe and cluster stories. This got expensive fast, so I'm wondering what your approach is to keep costs down across what I imagine is a lot of volume, i.e., one months' worth of stories across several sources.