simoncion

> ...is there a future where we end up with a version of the open web where everything isn’t paywalled or ID gated?

Absolutely! When OpenAI and Anthropic fail to secure enough funding to keep operating, the VCs backing the customers from which most of their money comes from will turn off the money faucets. This will dramatically reduce the amount of money being spent on the attack^Wover-exuberant scraper bots, which will drop that noise down dramatically.

Once folks aren't paying out the ass to serve up data for bots that really never use it, then a lot of the folks who have built or are building walls now will be able to tear them down.

Also, a great alternative to providing a government-issued photo ID and/or live biometrics is for Web Services to be obligated to use a government-provided categorizing scheme to categorize the things that their service provides and does, for major OS providers to be obligated to allow a local system administrator to set categories of things that certain accounts on that system cannot see or do, and then for the major web browser producers, people-who-publish-in-"app stores", and Web Service providers to be obligated to honor those sysadmin-configured restrictions.

This is really good because not only does it not require anyone to provide an ID and/or biometrics to some cut-rate third-party vendor, [0] but it allows -for example- both typical twelve-year-olds as well as dementia-damaged adults to be protected. The current systems pretend that every XXX-year-old is pretty much identical, and that zero people over the age of seventeen will ever need to be protected, which are obviously untrue things.

[0] ...who is 100% going to inevitably lose control of a large fraction of that data...

> A shop selling goods in $COUNTRY has very little to gain from allowing arbitrary users from any other country.

Eh?

a) Window shopping for the next time the out-of-country visitor is in the country.

b) An out-of-country visitor can contract someone who's in the country to purchase things from the shop and ship those things to them.

c) Queries and requests from out-of-country visitors can cause the shop to notice that setting things up so that they can ship internationally actually would be a money-making thing for them to do.

I do agree that the obscenely antisocial bots run for the benefit of the LLM Manufacturing Industry make it very hard for small-time operators to keep their websites up and running at any reasonable cost. Hopefully the major LLM Manufacturers will fail to secure enough funding over the next few months, their VC-backed primary customers will then have their money faucets shut off, and the attack bots will reduce down to a dull roar because folks have stopped paying "infinity" dollars for them.

> the actual value comes from ... [a whole bunch of automation] ... without it, users aren't going to have confidence in the continuity of the solver to build around it.

You're talking like this is a paid service that's attempting to get customers.

This is an MIT-licensed library on Github that's being offered free of charge. Sometimes people just offer what they've been working on up as a gift... one that is "...provided "as is", without warranty of any kind, express or implied, including but not limited to the warranties of merchantability, fitness for a particular purpose and noninfringement.", yanno?

> I’ll update my threat model to include sophisticated...

As sophisticated as your average low-rent startup IT guy, sure. "Can you set up an HTTP server with Basic Authentication and a TLS certificate?" is the bar here.

> I do, however, find it less likely a smaller scale non-state operation would bother hacking a government agency...

And yet, here we have someone who's not even profiting from the information who has obtained it.

> ... It was a blind spot.

Yes. Definitely. Whenever you find yourself thinking that only "sophisticated actors" and/or "well-funded actors" are motivated to and/or capable of poking around in Internet-facing services to see what they can find and/or do to them, remember all of the times that friggin teenagers and folks who meet some definitions of homeless have yanked dreadfully sensitive data out of said services.

I don't think we know the particulars of how this data was snagged, but so many of what businesses describe as "sophisticated attacks" by "persistent threat actors" are the meatspace equivalent of either walking through a doorway and making copies of everything sitting in plain sight in the room within, or unlocking a door with a well-labeled key that was sitting right next to that door, entering the room, and then firing up the universal copy machine.

Bonus chatter:

If Homeland Security agents who could be murdered, kidnapped, or both if their covers were blown could have their covers blown by data entrusted to friggin Okta, then Homeland Security wanted to have those agents killed and/or kidnapped. Don't believe this? Consider what would have to be done to make it possible for folks in a serious Federal Witness Protection program to earn a paycheck and pay taxes without alerting folks who would be rather interested in kidnapping and/or killing them and their families.

> My OS vendor didn't set up my firewall rules, lay out my subnets, or configure my routing. I did that.

Wow! Me too!

> Because I've got a slightly more advanced network layout than your typical home user.

Mine's quite a bit more advanced, if we're being frank. Though, the way I have things set up should be the default for SOHO stuff. This sort of configuration really is something whose setup can be automated, but never will be for as long as dumbswitches are the go to for SOHO equipment.

Anyway. I guess you and I just have dramatically different opinions on what's difficult. ;)

  2002:db8::1

Ah, so this is in the range assigned to 6to4 (2002::/16). Both Windows 10 and Android seem to treat it specially, [0] but Linux does not. So, I changed the AAAA record to 2400:db8::1 and re-ran my tests. I found out:

The 'getent ahosts' output remains unchanged.

When dealing with IPv4, all Linuxes, Android, and Windows 10 intermittently switch between preferring the GUA or the 1918 address. I seem to have gotten unbelievably (un?)lucky when testing v4 on Android, Gentoo Linux, and OpenWRT the first go around.

When dealing with IPv6, Android and Windows 10 always preferred the v6 GUA to the ULA. Every Linux I tried -and I tried several, each from different distros, some with and some without systemd, spread across years of releases- always preferred the ULA to it... except for OpenWRT, which always preferred the GUA. The only way I could get OpenWRT to prefer the ULA was to remove its GUA address but keep its v6 default route in place. Odd behavior. I wish I had a Steam Deck to see if "prefer the GUA" is something that appliance vendors tend to do, or if it's just a dubious OpenWRT customization.

The IPv6 behavior of all the OSs I tested is far better than the v4 behavior... predictability is always good. But it is remarkable that even many years ago, Linux was doing the sane thing for v6 address selection, while its commercial cousins and competitors -apparently- hadn't [1] bothered to think through how best to arrange things for "residential" users.

[0] ...even when no 6to4 addresses are configured on the host!

[1] ...and still haven't...

> Dear enemy - bombers maintained by shop X...

If the enemy could track back the bomber to the shop that maintained it based on the information in that report, they

a) Already knew that it crashed.

b) Already knew if any other bombers maintained by that shop had crashed.

As I said, bombers weren't routinely falling out of the sky. And as I suggested, releasing a declassified report that only covered the reason for the crash revealed nothing that the enemy didn't already know.

> Plus dual-stack means more work, and more possibilities for misconfigurations or vulnerabilities in your network.

Nope. Your OS vendor already did the work to make both versions of IP work. And as I mentioned, if you're using a decent firewall you can write IP-version-agnostic rules just as easily as you can ones that depend on the version.

Plus, most SOHO firewalls default to dropping unsolicited non-ICMP traffic. That's the exact same security stance that IPv4 + NAT gives you. You have to go out of your way (or be using fancy gear that is aimed at people who know what they're doing) to expose your LAN hosts.

> In IP discovery GUA is prioritized over ULA. And if the GUA goes away or changes regularly it leads to connectivity issues.

Sure, but you're generally not going to get both addresses associated with a single host unless you either hard-code them or use DNS. For any non-stupid network, source address selection for reaching LAN hosts doesn't really matter. And for many-to-most SOHO networks, you're going to be running dnsmasq, which will quickly set you right after any GUA change.

And like, the thing to be noticing here is that source address selection priorities for both IPv4 and IPv6 seem to give us the same actual behavior. Consider a pair of hostnames served up by BIND 9 and some applications running on bog-standard Gentoo Linux whose '/etc/gai.conf' contains only comments and blank lines:

  # getent ahosts dumbhost.invalid
  1.1.1.1         STREAM dumbhost.invalid
  1.1.1.1         DGRAM  
  1.1.1.1         RAW    
  192.168.30.30    STREAM 
  192.168.30.30    DGRAM  
  192.168.30.30    RAW    
  # ping -n -c1 dumbhost.invalid
  PING dumbhost.invalid (192.168.30.30) 56(84) bytes of data.
  From <redacted> icmp_seq=1 Destination Host Unreachable

  --- dumbhost.invalid ping statistics ---
  1 packets transmitted, 0 received, +1 errors, 100% packet loss, time 0ms

  # curl -v dumbhost.invalid
  * Host dumbhost.invalid:80 was resolved.
  * IPv6: (none)
  * IPv4: 192.168.30.30, 1.1.1.1
  *   Trying 192.168.30.30:80...
  * connect to 192.168.30.30 port 80 from <redacted> port 47140 failed: No route to host
  *   Trying 1.1.1.1:80...
  * Established connection to dumbhost.invalid (1.1.1.1 port 80) from <redacted> port <redacted>
  <snip>
  # getent ahosts dumbhost6.invalid
  fd00::1         STREAM dumbhost6.invalid
  fd00::1         DGRAM  
  fd00::1         RAW    
  2002:db8::1     STREAM 
  2002:db8::1     DGRAM  
  2002:db8::1     RAW
  # ping -n -c1 dumbhost6.invalid
  PING dumbhost6.invalid (fd00::1) 56 data bytes
  From <redacted> icmp_seq=1 Destination unreachable: No route
  
  --- dumbhost6.invalid ping statistics ---
  1 packets transmitted, 0 received, +1 errors, 100% packet loss, time 0ms

  # curl -v dumbhost6.invalid
  *   Trying [fd00::1]:80...
  * connect to fd00::1 port 80 from <redacted> port <redacted> failed: Network is unreachable
  *   Trying [2002:db8::1]:80...
  * Host dumbhost6.invalid:80 was resolved.
  * IPv6: fd00::1, 2002:db8::1
  * IPv4: (none)
  ^C
  # ip addrlabel list | sort -hk 4
  prefix ::1/128 label 0 
  prefix ::/0 label 1 
  prefix 2002::/16 label 2 
  prefix ::/96 label 3 
  prefix ::ffff:0.0.0.0/96 label 4 
  prefix fc00::/7 label 5 
  prefix 2001::/32 label 6 
  prefix 2001:10::/28 label 7 
  prefix fec0::/10 label 11 
  prefix 3ffe::/16 label 12

Notice how actual application behavior doesn't match what folks in this thread are saying that source address selection dictates. Android behaves the same way, and -weirdly- my OpenWRT hosts have the exact opposite behavior and prefer using global addresses to ULA or 1918 ones, regardless of IP protocol version.

Anyway. If he put both IPv4 GUAs and 1918 addresses in DNS or his '/etc/hosts', he'd be having the same problem he's complaining about here... depending on -I guess- the OS he's using and how his resolver treats the ordering of entires in '/etc/hosts'.

This isn't an IPv6 problem, this is a "Things break for a little while when I use addresses to communicate with a host that can change out from under me instead of addresses that I control and those addresses happen to change." problem.

> ULA work well in combination with DNS though.

Yep! Most everything works great with DNS. Remembering IP addresses is for the birds.

> I don't have a LAN's DNS server or a DHCPv6.

Yeah, that's why you think that this is a "puzzle".

Though, it does invalidate every complaint you've made about IP source address priority. The only ways you'll get multiple addresses for a given host are to either hard-code them, or use DNS [0]. Given that you're not using DNS, then you either have automation to update all those hard-coded configs, or you do it all by hand. Either way, you're deliberately recording addresses that you don't control for your LAN hosts, and considering it a "puzzle" when you have trouble reaching those LAN hosts when your upstream provider changes their address.

[0] Given that every resolver I've ever used pulls data from '/etc/hosts', I'm considering that to be part of DNS for the purposes of this conversation. I'm also considering Avahi and friends to be DNS. [1]

[1] And if you've talked to any long-term Avahi user, you've certainly heard about its habit of -on daemon or host restart- erroneously considering host name of the machine it's running on to be "in use" and appending a "-N" to the end of it. The remarkable eagerness of Avahi to engage this collision-avoidance mechanism makes it entirely unsuitable for discovery of LAN hosts whose hostname is actually important. This fact breaks my heart, but Avahi has been doing this shit for a very, very long time, so it's probably never going to get fixed.

> I believe the folks worrying about IPv6 only are just looking ahead to a possible future.

I'm certain that that is not what the folks who scream very loudly about how IPv6 has been and continues to be a colossal failure because it's not the only version of IP on the Internet are doing.

> I do think it is reasonable to expect there will become a time when IPv6 is pervasive enough, that new deployments will target users with IPv6 rather than IPv4.

Sure. That's the future I find most plausible: For "residential" connections, provide IPv6 GUAs and IPv4 via CGN... [0] for "commercial" connections, provide IPV6 GUAs and give the customer the option of one or more IPv4 GUAs.

> I will be very surprised if we don't hit that tipping point in my lifetime. Say the next 30 years.

I do hope we get to that point within the next five years, if not ten! But notice that my claim is that IPv4 will still be crossing the Internet after everyone reading HN today is dead. [1]

[0] I'm calling the various mechanisms that only provide an IPv6 address to an end user and do fancy gyrations to provide IPv4 access "CGN", even if might not strictly be CGN.

[1] ...as long as the the responsible, level-headed type of folks that run it now aren't replaced with the "making backwards-incompatible changes to everything every six months is A-OK!" type of folks.

> ULA are useless for most practical purposes because of the source address selection rules.

/me glances over at my LAN which has been happily using ULAs for local communications and GUAs for global for the past I don't even know how long.

_Sure_.

> Prefix withdrawal doesn't work reliably on Android.

Works fine on the Android hardware I have. Are you attempting to retire prefixes that have an infinite lifetime? That doesn't work, and I don't think it's supposed to work.

> Neither does this solution allow for load balancing...

As I said

  Or are you talking about load balancing, which is its own thing? Either way, this situation is inherently *not* boring, so the way you solve it is going to be quite a bit more complex -regardless of IP protocol version- than the very boring single-upstream situation.

> Printers get an RFC1918 address and don't care about global connectivity.

Okay, so why are you complaining about your printer being broken by GUAs that you're not permitting it to have? ;)

> IPv6 is nothing like IPv4 operationally.

For anyone who's not an IP stack developer, network hardware developer, or Enterprise sysadmin it absolutely is. And other than the fact that every endpoint is able to get a global address in v6, the sysadmin-visible differences between v4 and v6 are minor. RAs instead of... well, nothing, and DHCP for both.

> Whoops. The printing is now broken somehow. Why?

Exactly.

Why isn't your DHCPv6 server putting either both the GUA and ULA, or -better- only the ULA addresses in your LAN's DNS server? This Just Works™ on dnsmasq, which is one of the most widely-deployed DNS+DHCP servers on SoHo gear. [0] dnsmasq's default TTL for hosts added to the DHCP lease file appears to be 0, so -worst case- your sleepy computer tries to continue its conversation with the printer a couple of times, does ND and discovers that the IP it was using is gone, then looks up the hostname again and gets the new address of the printer. I think the trendy term for this back in the 2000s was "self-healing".

If you're going to retort "But we don't use dnsmasq in the companies I'm sometimes talking about!"... well, if you can't figure out how to not register upstream-controlled GUA addresses for your LAN hosts in your DNS, then do what dnsmasq does and serve them up with a TTL of 0.

[0] To head off the "Android doesn't really do DHCPv6" retort: Even though Android refuses to use anything other than DHCPv6-PD for managed address assignment, -AFAIK- Android still sends DHCPv6 ADDR_REG_INFORM messages for addresses it autogenerates on networks with either the "OtherConfig" or "Managed" flags set. I haven't bothered to set up my Linux systems to do this, but I'd be surprised if Windows and Darwin systems didn't do this.