Comment by promano

They also don't care which URL was accessed when the goal is scraping as much text as possible for AI training.