I'm not sure all companies will have deniability. It's pretty easy to see companies which are trying to evade detection or misusing their existing spiders for AI training purposes.
Some examples I've seen have crawlers which mask themselves 99% of the time by faking Chrome, but which will leak their real identity in UA strings which share the overall fingerprint of the swarm.
Some examples I've seen have crawlers which mask themselves 99% of the time by faking Chrome, but which will leak their real identity in UA strings which share the overall fingerprint of the swarm.
(I'm looking at you, Replit.)