I'm the administrator of 3 online forums, 2 of which are basically archives at this point. All 3 forums were getting absolutely hammered by bots. In fact, all 3 were taken offline by their hosts due to the heavy traffic in 2026. I installed Anubis on 2 of them and bot traffic decreased dramatically to the point where it appears to be only legitimate bots. (Unfortunately, the other forum couldn't use Anubis so easily and has remained offline since then.)
It's clear to me that Anubis won't work forever. You don't need to get as sophisticated as this blog post either as you can bypass the proof-of-work challenge with the right user agent string. I think the reason I'm not seeing bots change their behavior in response to me adding Anubis is simply that my forums are relatively small players. To the people running these scraper bots, a handful of forums adding Anubis is no big deal to them as there are tons of other sites they can scrape.
I think turning the forums that are archives into static HTML is the best path forward for those forums, but that's easier said than done. I've also been thinking about writing a new forum software where all of the content pages are static HTML, and the only dynamic pages are where dynamic features are unavoidable like when posting.
It's clear to me that Anubis won't work forever. You don't need to get as sophisticated as this blog post either as you can bypass the proof-of-work challenge with the right user agent string. I think the reason I'm not seeing bots change their behavior in response to me adding Anubis is simply that my forums are relatively small players. To the people running these scraper bots, a handful of forums adding Anubis is no big deal to them as there are tons of other sites they can scrape.
I think turning the forums that are archives into static HTML is the best path forward for those forums, but that's easier said than done. I've also been thinking about writing a new forum software where all of the content pages are static HTML, and the only dynamic pages are where dynamic features are unavoidable like when posting.