I’ve found that Claude is really helpful for finding patterns in my web logs and identifying bad traffic patterns. Meta was ignoring robots.txt on a wiki that I run and generating so much traffic that it was effectively DOSing the site until I blocked their crawler.
What’s been really handy is being able to identify “stealth” bot traffic by the request patterns (e.g., not requesting CSS/JS/favicon when loading the page).
What’s been really handy is being able to identify “stealth” bot traffic by the request patterns (e.g., not requesting CSS/JS/favicon when loading the page).