Comment by LinuxBender

Thankyou for this.

I make heavy use of HAProxy and use the silent-drop capabilities on my own personal sites. In addition to stick tables, I also use it heavily with acl's that pattern match on common attacks and enumeration.

My favorite stick table uses http_err_cnt to block enumeration attempts.

    stick-table	type integer size 127k expire 30m store http_err_cnt
    acl		abuses sc0_http_err_cnt gt 6
    http-response	silent-drop if abuses

I also find it useful to define known bot user-agents and IP addresses, then add conditional ACL's to enforce my robots.txt, since many search engines just ignore it.

    acl BOTS src -f /opt/.ha/bot.txt
    acl BOTS hdr_sub(user-agent) bash url get lack ing team oog lurp ook rawl bot map dback ltx pider scan bmast robe ithub hatweb ansci rchiv null isco =
    acl FILES path_end .html .pl .jpg .jpeg .gif .png .gz .xz .7z .zip .rar .m3u .ogg .mp3 .mp4 .avi
    acl FILES url_sub /post /upload /images
    acl PROT hdr_sub(host) site1 site2 site3
    http-request silent-drop if BOTS FILES PROT

There's probably a more cpu efficient way to do the above. It was just a quick hack to make robots obey and is not seo friendly at all. Maybe you have ideas how I could optimize that.