Comment by tracerbulletx

Why would that happen if the website owners are explicitly trying to prevent access? This isn't a technical issue, the agents are perfectly capable of working with the existing websites regardless of ads and javascript. The semantic web and machine readable representations was a 90s thing, agents don't need that really.