Comment by danradunchev

For everyone who miss the point.

- I don't want my content on those sites in any form and I don't want my content to feed their algorithms. So I do not care for opengraph or previews.

- Using robot.txt assumes they will 'obey' it. But they may choose not to. Its not mandatory in anyway.

- Yes, they can fake UA. This does not mean I should not take any measures to block them just becasue they can fake.