Buttons840

There was a guy in Missouri who pressed F12 (view source) and saw that the state website contained the SSN of all teacher--just right there in plain text in the HTML.

He reported it responsibly and then the governor personally started attacking him and threatening him with a felony.

I checked your link and I see that is a 2022 DoJ policy. How much peace of mind do you think a DoJ policy (not a law) from 2022 gives me?

I read as far as the first sentence: "The policy for the first time directs that good-faith security research should not be charged."

Soooo... the underlying message here is that good-faith security research is technically against the law, but their policy is not to press charges. They clearly consider good-faith security research to be technically against the law, because it wouldn't make sense to say "for the first time, our policy is to NOT charge people who follow the law"--that makes no sense. What they're saying is "we could charge good-faith security researches, but let's not as a matter of policy".

Reading further you see the policy is about NOT prosecuting people who do something like use an alias as their account name. That same paragraph says their focus should be on people who access systems without permission.

And to be clear, I am advocating for red teams that operate without permission. The bad guys are attacking our systems without permission, and I think the good guys should be protected when they do the same for a good purpose.

I'd feel a lot better if I knew some red team security researchers were going hard trying to break into critical national infrastructure, just to ensure it's not too easy, but probably this isn't happening because it would be a felony to even try. The bad guys are free to try though.

All this will tend to get blamed on AI, but AI didn't put these vulnerabilities into place. AI is just a tool that can be used to discover vulnerabilities, and the only question we have to answer is whether or not that tool should be available to regular people.

Some will try to frame the question as being about whether we should allow AI aid in cyber-security, but that's not the question. This technology isn't going away. The bad guys will have it. Nations will have it. Again, the question is, can regular people also continue to have it?

And this is a society where many of the most powerful corporations got started by just blatantly ignoring the laws of their industry. See: Uber, OpenAI, etc.

Is an AI that is operating a browser on my behalf to answer a question I asked 5 seconds ago--is that a bot or a person?

Building on your analogy: Currently we spend a lot of time and effort making sure our walls appear nice and inviting, with lots of convenient entry points where advertisements can be placed. In the future we will build ugly walls with a few well guarded entrances.

What I mean is, look at how much time and effort we spend on making software services beautiful and branded, and so little time spent on security. In the future we'll probably have fewer webpages and more hardened text-based APIs that end users interact with via their AI agents.

Oh yeah, Screeps, that game is fun and I might stick with it this time if I can vibe code and turn my ideas into code without spending a lot of time.

I'm thinking it might be a nice playground to just experiment with different game AI architectures.

I've been thinking of trying to combine extremely simple Machine Learning techniques, like tabular Q-learning, with a variety of scripted action and goals. I want to try mixing a lot of really tiny machine learning algorithms (maybe neural networks, but not necessarily) into classic AI scripting and see if it can be more dynamic, but still predictable and controllable. Screeps would be a great testing ground.

I've always thought an evil Skynet based on language models is kind of a funny idea, because all of it's secret plans are just spelled out in plain text.

That's a strawman argument. I'm just saying that we need to find ways to wrangle the mountains of code coming out of LLMs into something we can trust, as best we can.

We need to build systems where trust in the code is not needed.

I don't trust the JavaScript code I run in my browser, I don't need to.

A language with a strong enough type system and effects system could say, "this function updates the users preferences, and may not access the network, and may not access data from other users", and the compiler can ensure these constraints AT COMPILE TIME! Provably correct code.

"Illegal" and "shouldn't be allowed" are the same thing when it comes to the government.

If something is legal, the government should allow it.

I am saying that it shouldn't be allowed, because it is not in harmony with the laws, and is therefore illegal.

If the government is going to get involved here, I would regulate companies like DoorDash or Uber to require them to provide company vehicles to drivers.

When DoorDash drivers subtract vehicle wear and tear, fuel, depreciation, and maintenance from their earnings, their net hourly pay frequently drops below the standard minimum wage.

DoorDash benefits from desperate people who slowly transform their vehicle into cash by laboring thousands of hours. They could have just sold the car; either way they end up with cash but no car.

People need to pay rent, so they look at their capital, owning a car, and then trade away that capital for less than it's worth, and a few years later they are worse off, but at least they made rent for those months.

Monopolies are not illegal, anti-competitive practices are illegal; there's a subtle difference.

You're right. Duopolies aren't great either and just because two companies are in fierce competition doesn't mean everything is good with the law. Both companies might be engaging in illegal anti-competitive behavior.

The AI should have a tool that it can use to quote an underlying document. The AI itself would not be responsible for filling the quote; that would be done by the interface which deterministically quotes the requested section.