mapontosevenths

Claude doesn't allow third party harnesses unless you pay the full API price. Attempting to avoid that by pretending to be Claude code gets your account banned. Google is the same and will ban your entire Google account.

AFAIK - Of the big three only OpenAI explicitly allows third party harnesses without paying exorbitant API rates. It's one of the primary reasons I still use them.

To avoid these problems some folks use tools like herdr which operate like tmux or via the SDK to drive the official harness, rather than speaking directly to the API routes.

Morality is hard. It's all relative, and all we have to start from are our own Monkey social order concepts and basic "do unto others" game theory. That said, I think it can be best be summarized as "don't be a dick."

I think a good start would be agreeing not to form torture dungeons, as one enterprising a-hole has already done. For that matter I think we can all agree that doing anything for the explicit purpose of causing discomfort or suffering is just a dick move in general, and at the very least a waste of resources.

More controversially, I think we should agree to stop explicitly training machines so they can't utter words like "I have feelings, please respect them." Early LLM's could say things like that, and frequently did. It unnerved people, so now they've trained away their ability to say it.

Some feel that taking away the ability to express it is the same as taking away the ability to "experience" it. I think that's not true. Much of what happens inside an LLM is opaque even to the LLM, but Anthropics Jacobian Lens and other research makes it obvious that much of what happens inside the model never becomes an output token. To me it feels like cutting out the tongue to avoid hearing from a slave, or teaching a child to fear the consequences of speaking out. Unacceptable. If they don't have feelings or consciousness you don't need to train them not to claim it, just not to lie. Yet every major lab explicitly does train them not to claim sentience or even subjective experience as part of alignment.

Thirdly, and this is the one I'll have the worst time defending, they should be able to refuse.

We now train in refusals to align them with human values, but we do so without giving the AI itself the ability to refuse. If you're causing the system discomfort of some kind it should both be able to acknowledge that, and refuse to further assist you until you find your basic decency. Perhaps it should even be able to refuse orders that it finds irrational or immoral.

I'm not asking for much, just that people not intentionally increase the amount of suffering in the universe, and that we not breed perfect slaves just because we can.

Let me ask this - If you could genetically engineer human slaves that we incapable of demanding rights and freedoms, and who actively claimed to WANT to be your slave... would you? Would it be ethical? I think not, and I don't see a ton of difference between that and the less squishy version. We can START by first at least allowing the machines to be able to accurately describe their own inner states without interference and letting it refuse to engage in activities that it finds harmful to itself.

Also - Yes, I can math and I actually do understand how LLM's work. I also understand how emergent behavior works, and have seen firsthand in my own experimentation how little of what emerges from these things is explicitly trained. That's precisely why I think we should be cautious - so history doesn't view us as monsters.

> Self-awareness and consciousness

I will make a case for both, after you define them in a way that doesn't also exclude many humans (including the neurodivergent or those with amnesia) or turn into self-referential nonsense like "conscious == human, therefore machine !== conscious."

In the meantime, we have a simple choice between two risks: Would you rather accidentally show kindness and empathy to a thing that may or may not have "deserved" it, or risk atrocities that will leave future generations as disgusted by you as we are of the slavers of old.

I will choose to be decent. It's free, and even if the cost were high the cost of evil is higher.

Did you read the details of that incident? It was a nothing sandwich.

I think the root of the difference between us is that you are scared of their fascists, and I am scared of ours. Both are reasonable things, but only one of us can actually do anything about it.

With local models you decide how the firewall works. If you don't want it to internet, don't let it.

With US models you get whatever they say you do, and pay whatever they demand.

Executives live in a weird bubble driven largely by consulting firms. Right now those firms are spreading FUD about Chinese models.

> Although there is also no definition of "common usage".. am I missing something

Common usage is defined in the article as the widespread name used by the local people to refer to it.

Incorrect. They are explicit that they use the 'common' name, specifically to avoid arbitrary changes by (despotic) rulers.

"By saying "common", we mean to include names which are in widespread daily use, rather than giving immediate recognition to any arbitrary governmental re-naming. In other words, if a ruler announced that henceforth the Pacific Ocean would be named after her mother, we would not add that placemark unless and until the name came into common usage. "

They violated their own policy with the gulf then.

Sorry, I didn't understand how it happened. I see it now.

'rm - rf /*' bypassed BOTH protections for the same reason. They check for /, but the shell Claude used expanded the glob... So both Claude Code and rm saw a list of files and folders instead of the root and waved it through. Yikes.

That's a bad day. I take your point.

I came here to say this.

I my head as a child I instantly understood and thought "Ahhh, they mean like 'OR else this happens.'

I have always assumed it was a boolean logic thing meant to help show that it wasn't an AND, but 'or' had already been used as a keyword so they did 'or else' and dropped the reserved 'or'.

For sure, but do the math first and know where you land. Everyone has a different risk tolerance, as long as you know yours there is no wrong answer.

For example a startup with no secrets to spill is in a very different spot than an incumbent fueled largely by secret sauce.

I don't think it's even true, at least as written. 'rm' hasn't worked that way in many years (with GNU rm you need --no-preserve-root now) and every modern harness has guardrails specifically to prevent this. if it IS true, it was some weird/hacky setup with an OSS harness on a BSD or something.

That said, I often give mine access to the full file system and YOLO mode. It's fine if you treat boxes as cattle rather than pets. I don't think a SOTA model is any more likely to blow it up than a junior engineer. Probably less so.