mquander

The linked report seems almost useless -- it doesn't say anything about an error rate or a sample size, so it's a mystery whether 9 out of 20 systems “fabricated information and made suggestions to patients' treatment plans” ten out of ten times, or one out of a thousand times.

If we just postulate that the systems have a high error rate, I wonder why they are being adopted. They seem extremely easy to test, so I don't see why doctors or hospitals or governments should be getting tricked into buying them if they suck.

I haven't, but the parent said that this is what a company "should" do, not just what they do do.

> Of course the safest (first) option is the correct option from a liability standpoint, which is all a company should operate on since it's first responsibility is to protect the company for those that are still there.

Isn't this an unrealistically black-and-white mode of thinking? Humans are complicated and have many values and perceived responsibilities. It's not healthy for them to throw them all out and act as if they only have one responsibility that needs to be maximally upheld at all costs. They should balance their actions thoughtfully.

That rule is a rule that excludes many socially excluded nerds. So if there are more and more spaces like that, then the nerds that those spaces exclude, are probably going to go make new communities and attempt to keep out the people who just excluded them.

It doesn't exclude people who attend high schools and colleges that have a computer lab.

Your comment is framed like "giving a student a personal AI datacenter to carry with them" is unrealistic, but in fact it is easy for anyone with access to $1000-$2000 worth of compute to download and operate exactly that for free, with performance perhaps a year behind the state of the art.

This comment would make more sense if it were before the new wave of prediction markets, which are high-profile gambling products clearly largely made and popularized by true believers who think they are making the world a better place.

I think it's slightly less ridiculous than it sounds, because governments have much more power over their own citizens. As an American I would dramatically prefer the Chinese government to spy on me than the American government, because the Chinese government probably isn't going to do anything about whatever they find out.

(That logic breaks down somewhat in the case of explicitly negotiated surveillance sharing agreements.)

> I don’t know who operates this agent, and I’m not going to speculate about why they did what they did.

That's great, I'm going to use that one in the future.

I recommend Matthew Sadler's Game Changer and The Silicon Road To Chess Improvement.

I don't know if this matters to you, but Eliezer doesn't think LLMs will kill us. He thinks LLMs are a stepping stone to the ASI that will kill us.

This is fascinating, thanks so much for posting it!