Comment by scellus

Are citation issues related to the fact that https://www.bbc.co.uk/robots.txt denies a lot of AI, both user agents and crawlers?