If someone tells you to go away via the robots exclusion standard, and puts up bot mitigation to prevent you, blocks your IPs, etc. then clearly you do not have their consent to help yourself to the data.
I find it really hard to see how you could twist ignoring this clear lack of consent, and going to great lengths to circumvent what was clearly put into place to prevent you from doing the very thing you are doing, how you could twist that into an ethical action.
It may or may not be technically illegal to do, you're but that is not a statement about what is ethical.
I find it really hard to see how you could twist ignoring this clear lack of consent, and going to great lengths to circumvent what was clearly put into place to prevent you from doing the very thing you are doing, how you could twist that into an ethical action.
It may or may not be technically illegal to do, you're but that is not a statement about what is ethical.