Fine, but you have the invest in the CPU to determine if the payment was made. That's a cost on you, not the scraper. If the scraper bangs on that payment door 1000x you've spent a fair chunk of CPU looking for valid cookies, query the redis/valkey and DB to manage payment auth.
You're not wrong, it is time the scraper pays, but you're going to have to fund it still.
I don't claim to have a better answer.
I want a perfect solution that is free, putting pages up used to be free, now it isn't.
You're not wrong, it is time the scraper pays, but you're going to have to fund it still.
I don't claim to have a better answer.
I want a perfect solution that is free, putting pages up used to be free, now it isn't.