Comment by Schnurpel

You can get the originating IP via mod_remoteip, or its nginx brethren. You can block those IPs in your firewall, or via the Cloudflare firewall.