RobSm

You can always stop bots. Add login/password. But people want their content to be accessible to as large audience as possible, but at the same time they don't want that data to be accessible to the same audience via other channels. logic. Bots are not consuming your data - humans are. At the end of the day humans will eventually read it and take actions. For example chatgpt will mention your site, the user will visit it.

And no, nothing was different before 2022. Just look at google, the largest bot scraping network in the world. Since 1996.

The reason is different. When you need to scan 5 million IPs on 22 port is one thing, but 5 million IPs times 100 or 1000, then you simply run out of resources and since 99% of people use 22, why bother?

There are at least 1 of us!

Looks weird :)

You need to upgrade your knowledge. Use only [2-9]. That's it.

Google scrapes everyones data yet you use Google every day and find it useful.

No

People don't care about such content and so does Google.

Explain more about it

And if I open the website in my browser and then copy from browser to my computer, then all is good?

And if you manually copy someone's data they worked hard to generate to go and resell, then it's ethical?

This is so exactly. People do not realize that when they use chrome to view website, chrome is their 'scraper'.

And the goal of webs craping is not to get illegal data, but to have efficiency and performance by not doing something manually but letting computer do the repetitive tasks. It's a productivity tool. You can't make something illegal just because it's an automation instead of 'manual' operation.

No. robots.txt is not something that is defined and enforced by the law. Just because someone came up with some 'recommendation' like robots.txt does not mean this is the law

If you post that data on a public domain, that is publicly available. It's like writing that info on a cardboard and putting it in the town square and then saying 'why you people steal my data!'

How many contracts google breaches scraping billions of pages every month?