Comment by cabirum

The crawlers can simply stop identifying themselves via custom user agent, can't they?

Also why are "AI" crawlers are worse than "normal" crawlers?

Either way, this is an exercise in futility.