Comment by godrae369

Hey! Saw your post about GPTBot eating up your Cloudflare Workers quota. The brutal truth is that relying on robots.txt in 2026 is like putting a 'please do not enter' sticky note on a bank vault. AI scrapers are notoriously ignoring them or rotating IPs. You are literally paying Cloudflare so OpenAI can train their models. I build custom infrastructure defenses for SaaS founders. Instead of hoping they respect your robots.txt, I can set up specific Cloudflare WAF (Web Application Firewall) rules and Edge Workers that fingerprint AI scrapers (even when they spoof user-agents) and drop the connection at the edge before it hits your billing quota. If you want to permanently lock them out and protect your server bill, let's chat. I can share the JSON ruleset for your Cloudflare dashboard.