ag8

It's incredibly robust to fine-tuning; often gradients don't flow through the digital locker, and even if you get this to happen, the amount of noise you can add to the weights of these cryptographic circuits before they stop working is often orders more than the amount of noise you can add to the language modeling parts of the language model before that stops working! So in short, this is often more robust than the part of the model that knows English. Same thing with abliteration; there's no "direction" in weight/activation space that you can subtract here.

Right, but access to the training pipeline and intermediate checkpoints can allow us to catch the development of these structures in model weights before they become fully undetectable! It's a bit like monitoring agents communicating over noisy channels: it's interpretable for a while, but as soon as they come up with asymmetric key exchange, anything that looks to you like "noise" might actually be the agents scheming. So it's important to catch this _before_ it occurs!

Yeah. Also, we think that in certain scenarios, it's possible for "cryptographic patterns" to arise in model weights via normal training, especially if models are trained against things such as honesty probes.

I find this paragraph to be odd: "Wavelengths as low as 13.5 nanometers can achieve more precise patterns in a single exposure. In fact, extreme ultraviolet lithography can combine three or four photolithography patterning cycles into a single one on a seven-nanometer node. Without EUV, producing five-nanometer nodes might require as many as one hundred different steps."

But a five-nanometer node has a gate pitch of 45nm and a metal pitch of 20nm! Using different forms of the word "nanometer" in the same paragraph is very confusing...

You're right; I should've been more precise. However, we have tools for dealing with this—that's what quality-adjusted life-years are for! I don't contest that surgeries often significantly increase QALYs, and may do so pretty cost-effectively.

Lol, I just care a lot about saving as many lives as I can; the most effective charities I've been able to find good evidence on save one life for $6–8k. If Watsi had a credible claim at being able to save lives 10x cheaper I would redirect my entire donation budget to them!

That said, once again, Watsi is great. I really appreciate all the hard work they've put into making this happen—this is orders of magnitude more impressive and impactful than most projects I've ever seen!

Watsi seems to be doing great work, but the title—"you helped save 33k lives"—reads as misleading to me. I guess "helped" could be doing a lot of heavy lifting here, but I would be incredibly surprised if the counterfactual number of lives saved was more than 3000. (But don't let this dissuade you from donating; concretely improving someone's life is totally a worthwhile goal, and Watsi seems very good at effecting this)

Not 13?