Kodiack

Why yes, 10-minute-old account with random letters for a name and one single comment, both sides are absolutely the same and people just shouldn’t feel encouraged to vote am I right?

You’re not even remotely subtle smh

I use Octopus in NZ and I’m a big fan of what they’re doing compared to a lot of our other electricity retailers. Savings sessions in particular are great with a beefy home battery because exporting counts in the comparison average. Those are easy money-making opportunities for us.

LLMs are incredibly powerful and useful tools, but offloading your writing to them oftentimes means you’re placing an undue burden on whoever you’re sending the generated output to.

To take it to the extreme, a “3x worker” is by definition relative, and someone malicious could achieve that by overloading a colleague with generated text that they themselves hadn’t bothered to read in full. You can stay at your baseline but be “3x” if you’re sufficiently able to sabotage your coworkers.

I’ve been on the receiving end of needing to review “documentation” that was generated almost entirely by LLM. It wasn’t anything that was done with malicious intent, but it’s still exhausting, and you need to exercise an even greater degree of caution because of the potential subtle mistakes or inaccuracies that a real human would never make.

I run some small websites, including a tiny forum that’s been a goldmine for scrapers. I had to significantly tweak some firewall rules and configuration after scrapers behind residential proxies suddenly accounted for over 99% of requests.

However, I also relaxed rules for automated traffic that was well-behaved, and I went out of my way to ensure that the Wayback Machine was able to hit everything. I should kick a small donation their way. They provide an incredibly valuable service and I love the benefit that I get from them just for personal side projects.

As someone who hosts a few “tiny websites” that are “hiding behind Cloudflare”, it’s because of some incredibly misbehaved botnet traffic that’s otherwise persistently scraping via residential proxies.

This is a hobby for me. It’s for my enjoyment and it allows me to provide resources that others enjoy using. However, I’m not going to allow literally 99%+ of requests to be aggressive scrapers that won’t give up until you’ve got a heavy-handed solution in place.

I hate it too, but the alternative is even more consolidation, so unfortunately this is just the reality right now and you’ll have to get over it until when/if things improve.

Noise and air pollution are valid concerns with a lot of these datacentres being built as well. Some of these recent datacentres have been built close to residential areas, and the health harms are far more worrying than the property valuations IMO.

I use my home connection for VPN access remotely. I back up snapshots of data every day. I like to be able to download games and Linux ISOs practically on demand. I work from home and often enough faster speeds can avoid several minutes of additional waiting in a day.

This connection is shared as well. My partner relies heavily on cloud syncing. We both like to stream 4K HDR video. I like being able to get devices updated and ready to use with minimal time spent waiting for downloads.

I also live in NZ, where multi-gigabit fibre connections are often cheaper than what Americans have to pay for a fraction of the bandwidth. It’s not a notable financial burden or anything, and it’s not like we have data caps to worry about. It’s very much a situation where the use cases naturally find themselves once the option is there.

Also, 25/10 Mbps is painfully slow for a shared home connection in the modern day. There’s videos on YouTube that can push a higher bitrate than that. The absolute slowest plan that my ISP even offers is 100/20 Mbps for about $35 USD per month, while the most common/baseline plan for most households in NZ is 500/100 Mbps after the fibre carriers continued to increase speeds at the lowest tiers.

I love seeing trees in more places, but for parking lots in particular they do have some downsides compared to solar panels. They often require more space; they attract birds that that poo on vehicles; and there’s a higher risk of collateral damage during windstorms. Not to mention that solar panels directly produce electricity, of course.

We absolutely should see more trees in many cities, but they introduce their own challenges in parking lots, especially if they’re placed retroactively.

The extent of that heavily depends on where you are. Where I live in NZ, the grid export rates are very low while the import rates are very high.

Our peak import rate is 3x higher than our solar export rate. In other words, we’d need to sell 3 kWh hours of energy to offset the cost of using 1 kWh at peak.

We’re currently in the process of accepting a quote for home batteries. The rates here highly incentivise maximising self-use.

If you’re using Linux you’re almost certainly using your package manager to get any relevant updates.

Automatic updates are absolutely not peak stupidity. Most users’ devices would have nasty security vulnerabilities wide open for a much longer period of time without automatic updates.

I was thinking the same, and read it as “Xi”kipedia. Then sure enough, one of the articles that it immediately showed when it loaded was for “General Secretary of the Chinese Communist Party”, or Xi Jinping.

Coincidence?! Yeah, probably.

Hey. I run a small community forum and I've been dealing with this exact same kind of behaviour where well over 99% of requests are bad crawlers. There used to be plenty of "tells" for the faked browsers, HTTP/1.1 being a huge one. As you said, however, they're getting a bit smarter about that and it's becoming increasingly difficult to differentiate it from legitimate traffic.

It's been getting worse over the past year, with the past few weeks in particular seeing a massive change literally overnight. I had to aggressively tune my WAF rules to even remotely get things under control. With Cloudflare I'm aggressively issuing browser challenges to any browser that looks remotely suspicious, and the pass rate is currently below 0.5%. For my users' sake, a successful browser challenge is "valid" for over a month, but this still feels like another thing that'll eventually be bypassed.

I'd be keen to know if you've found any other effective ways of mitigating these most recent aggressive scraping requests. Even a simple "yes" or "no" would be appreciated; I think it's fair to be apprehensive about sharing some specific details publicly since even a lot of folks here on HN seem to think it's their right to scrape content with orders of magnitude higher throughput than all users combined.

I really don't know how this is sustainable long-term. It's eaten up quite a lot of my personal time and effort just for the sake of a hobby that I otherwise greatly enjoy.

If you have a "legitimate scraping pursuit", identify yourself appropriately that way. I'm happy to let most well-behaved scrapers access my content.

Hiding behind a residential proxy and using random user agents? Gross. Learn what consent is.