ChrisHardie

The usual case I run in to is that a site will block requests with User-Agent header strings that don't at least try to look like a regular browser, or that appear on some list of known bots/automation tools. (If they are using Cloudflare, this is a very easy state for a site to get in to.) I'm not sure if GH actions lets you customize the user agent in the spot you're hitting the issue, but that's where I'd start.

Once I created the structure to support lots of different kinds of scraping-to-feed conversions, it’s usually fast to add a new target site in to the mix. There are definitely exceptions, and definitely the occasional maintenance when someone updates their CSS.

Articles should not have images that are misleading or confusing, but I do understand why most news articles have something when it comes to imagery. Most news website designs are optimized for at least one image per article, and social media sharing almost requires it if you expect any kind of engagement at all. But it’s not a problem the media world should have to solve at the expense of the reader experience. (Disclosure: I’m a journalist and digital editor who spends way more time than I’d like trying to pick the least harmful stock or file image in the cases where we just don’t have a good image for a story.)