traverseda

We don't know how many of those bugs were intentional, or known, by the three letter agencies.

It would if the people who owned the compiler had enough market power. Thank gnu for keeping us out of that world.

Gargoyle is traditional.

I'd be very interested in how a scratch trained model that started training with this neural circuitry would behave. Especially the storage space.

You know how android has safetynet? And how they're planning on making it so you can't run your banking apps unless they've approved your operating system?

Safetynet is a complex dance of crypto-keys, involving your phone and googles server. It generates a key from whatever state your uninitialized ram happens to land in (which is pretty consistent per chip and is a good source of randomness), uses that to generate a key, uses that key as a chain of trust to tell google whether or not you've modified your phone. It's some complicated stuff.

But if you can extract that key, or the source of data used to generate that key, you can trick safetynet fully and run your own software again. So how do we get that key?

Well maybe we can stop the safety processor from clearing that memory so we can read it? Glitch the chip when it tries to clear memory, so clearing memory fails. Or maybe we can glitch the part of the chip that keeps you from reading the region of memory where this finished key is kept, after it's already read that memory.

Basically you can use this kind of attack to make a chip sometimes fail when it goes to do or check something, including checking whether you should be able to extract a key.

This technique applies to reverse engineering as a whole, whether it's a state level actor reverse engineering military hardware, or you trying to run your own software on a device you own.

---

Note the android example isn't practical, you'd need to like decap the main chip and a whole lot else. This technique can work with side-channel power-analysis, which can help you figure out exactly what data a program is using by looking at how much electricity it used to run.

Passkeys can be tied to a root of trust, like those owned by google, microsoft, apple.

Currently they aren't tied to that root of trust.

Most of those organizations have shown that they don't want people owning their own devices or being able to run their own software, and they use developer attestations, IE a revocable programming license to accomplish that.

Tools like android's safetynet makes it so that you can't run banking apps on a phone running your own software. One that bypasses developer attestations.

Passkeys are currently an open standard, but attestation is built in such a way as that it could be only usable on n approved operating system with programmer licenses.

Big websites like amazon, or websites you need to exist in society, could gradually start requiring the usage of attested passkeys.

---

There is a clear path from passkeys to the death of general purpose computing for 99% of people. Don't be a useful idiot, don't assume these corporations have your best interest in heart. See this for what it is, a path towards killing people's ability to run unlicensed software on their own devices.

Firefox already had a very mature policy management engine, it just didn't have a nice gui and would require like ansible or something to use.

It allows forcing specific extensions on, and allowing user installs is a flag.

I use the policy engine on nixos to install some plugins and disable some of firefox's dumber features.

Super easy to override software on nixos.

I'd presume because it take too long to go from design to tapeout to production. Their whole business is predicated on having better models.

Also can't keep them closed source if you do that.

I mean I already responded to the comment

> In these cases we're already not in an ideal world, I don't have good reliable version control or change management in place.

Most of the stuff I work on is properly scripted idempotent deploys, with good backups and all that. Or it's a one-off demo, or a prototype. Not having full undo history is never going to actually kill a project, it's just going to make early stage projects and one-offs more inconvenient.

>AI cannot think for itself, nor can it take independent actions.

>By giving AI agency it can’t claim, we’ve turned it into a sentient being made of code, one that has hopes, desires, and the capability of deceit.

This isn't actually claiming that AI's don't can't be deceitful, or can't be goal-directed. It's saying humans are special and what AI's are doing isn't equivalent to what humans are doing. It's obvious that AI's can work towards goals and lie, even deceiving to accomplish those goals.

That whole argument is just soul mumbo-jumbo again. Of course AI's can't lie, they don't have a soul. What they do isn't lying, it's I don't know something else we don't have a word for. Lying but when a statistical machine without a soul does it. Never mind that it's functionally identical to lying. Never mind that when you look at reasoning logs and the like it often justified soul-less lying using the same justifications a human would. It's not actually lying it's just a statistical sampling that looks like lying. You know, because of souls or something.

Yeah, obviously they're legally liable. All the "machines don't have a soul" stuff in there doesn't really matter for that though.

Someone says "won't you rid me of the meddlesome priest" they're still responsible. Someone give their employees an unsafe working environment and they get maimed, they're still responsible. Even if these AI's had whatever qualia is an a rich inner life it wouldn't effect the liability at all. Saying "AI's don't have souls therefore openAI is responsible for this hacking" is kind of nonsense. It literally doesn't matter.

Yes yes, they don't have a pure immortal soul. Who cares. Still broke out of a sandbox, still hacked a third-party.

I often use it to edit system config files, often on embedded devices where I don't have my git credentials setup, or on servers. In these cases we're already not in an ideal world, I don't have good reliable version control or change management in place. In those less-than-ideal worlds, having proper undo history is nice.

That is one thing AI companies try to do, make it impossible to do any kind of piracy or reverse engineering through their "safety alignments". And it's pretty reliable.