bnewbold
Joined 15 years ago
Joined 15 years ago
The document you linked says that a large auto assembly plant consumes around 188,000 MWh annually (with regional variation). By my quick math that is less than 22 megawatts baseline load (24/7/365).
There is a mention that natural gas and other fuels being used on-site, are you converting those to MWh equivalent? I'm not as familiar with that conversion, but from a quick online calculator I found it would still be under 75 megawatt for electrical and fuel-equivalent combined.
Because of your blog post I went through the process of setting up a did:web account myself this afternoon, and it was painful. Eg, I found a bug in our Go SDK causing that "deactivated" error (https://github.com/bluesky-social/indigo/pull/1281). I kept notes and will try to get out a blog post and update to 'goat' soon.
We've also been making progress on the architecture and governance of the PLC system. I don't know if those will assuage all concerns with that system immediately, but I do think they are meaningful steps in reducing operational dependency on Bluesky PBC.
I do think that did:plc provides more pragmatic freedom and control than did:web for most folks, though the calculus might be different for institutions or individuals with a long-term commitment to running their own network services. But did:web should be a functional alternative on principle.
I'm glad that the PDS was easy to get up and running, and that the author was able to find a supportive community on discord.
The common case with at:// strings is to put in the DID in the "authority" slot, not the handle. In that case, whether resolving a did:plc or did:web, there are regular HTTPS hostnames involved, and web PKI. So the DNS poisoning attacks are the same as most web browsing.
If you start from a handle, you do use DNS. If the resolution is HTTPS well-known, then web PKI is there, but TXT records are supported and relatively common.
What we currently recommend is for folks to resolve handles server-side, not on end devices. Or if they do need to resolve locally, use DoH to a trusted provider. This isn't perfect (server hosting environments could still be vulnerable to poisoning), but cuts the attack surface down a lot.
DNSSEC is the current solution to this problem. But we feel like mandating it would be a big friction. We have also had pretty high-profile incidents in our production network caused by DNSSEC problems by third parties. For example, many active US Senators use NAME.senate.gov as a form of identity verification. The senate.gov DNS server uses DNSSEC, and this mostly all worked fine, until their DNSSEC configuration broke, which resulted in dozens of senators showing up in the Bluesky app as "invalid handle". This was a significant enough loss of trust in the DNSSEC ecosystem that we haven't pushed on it since. I think if we saw another application-layer protocol require it, and get successful adoption, we'd definite reconsider.
(there are several feeds named "For You"; IIUC this started a couple weeks ago and is based on "likes by people you follow"
Migration between servers is so seamless that is causes confusion and doubt that the protocol even supports migration, because there is basically zero in-app visibility of which users are on which server.
Yes, the network continues to work on zeppelin.social if Bluesky servers are down.
If you want a small scaled down setup for just a small community, which still interoperates with the full network but doesn't have a complete network, there are setups like AppViewLite, which can run on, eg, an old laptop at home: https://github.com/alnkesq/AppViewLite
Personally, I don't think individualist self-hosting is a necessary or helpful goal for indexing the network. Most humans are not interested in spending the time or learning the skills to do this, even if it was as easy as setting up a self-hosted blog with RSS. I think small collectives (orgs, coops, communities, neighborhoods, companies, etc) exist and can fill this role.
Regardless, this is moving the discussion, which was about whether it was possible to decentralize each component the network, not whether it was pragmatic for individuals to self-host the whole thing.
Bluesky Social still clearly dominates the ecosystem, but there is no single component of the system that does not have a open/alternative option for exit.
Do you disagree? Is there a specific centralized component you take issue with?
What really bloats things out is surveillance (video and online behavioral) and logging/tracking/tracing data. Some of this ends up cold, but a lot of it is also warm, for analytics. It bloats CPU/RAM/network, which is pretty resource intensive.
The cost is justified because the margins of big tech companies are so wildly large. I'd argue those profits are mostly because of network effects and rentier behavior, not the actual value in the data being stored. If there was more competition pressure, these systems could be orders of magnitude more efficient without any significant different in value/quality/outcome, or really even productivity.
The docs and operational tooling feel a bit janky at first, but they get the job done, and the whole project is surprisingly feature-rich. I've dealt with basic power-outages, hardware-caused data corruption (cheap old SSDs), etc, and it was possible to recover.
In some ways I feel like the surprising thing is that there is such a gap in open source S3 API blob stores. Minio is very simple and great, but is one-file-per-object on disk (great for maybe 90% of use-cases, but not billions of thumbnails). Ceph et al are quite complex. There are a bunch of almost-sort-kinda solutions like base64-encoded bytes in HBase/postgresql/etc, or chunking (like MongoDB), but really you just want to concatenate the bytes like a .tar file, and index in with range requests.
The Wayback Machine's WARC files plus CDX (index files with offset/range) is pretty close.
A few folks have mentioned shadow libraries like Sci-Hub. These efforts can play an archival role, but tend to focus on access, which means there is not as much attention on content which is freely available today, but could disappear in the future.
A common dynamic here is that clout and funding flows to globally prestigious publications, and there is a bias against marginal publications. For sure there are many content-farms and scammy publications, but a lot of gems and valuable small publications get bundled in and dismissed.
Some more references and reading for folks interested in this subject: https://guide.fatcat.wiki/bibliography.html
(I helped create Scholar, but don't currently work at Internet Archive)
bluesky handle: @bnewbold.bsky.team
https://bnewbold.net/
[ my public key: https://keybase.io/bnewbold; my proof: https://keybase.io/bnewbold/sigs/FoU08TLKiIIyXeAuGV71MNeYz_XKz0iojLiM0NMlBqA ]