spacebanana7

Unions also require the right structural conditions.

For example waste management generally has very strong unions in the developed world because 1) capital intensity 2) operations immediately seize on labour withdrawal 3) consequences of strikes are in a golden zone of painful but not existential so are tolerated 4) extract a relatively small proportion of local gov funds, so don’t create that much powerful opposition 5) labour force is locally concentrated and highly socialised.

Those pillars of power don’t rely much on any particular legal or political system.

Imagine prison workers that were usually kept in solitary confinement were allowed to choose their uniform, and knew guards would suggest all prisoners adopted the uniform of high performing workers.

These prisoners could, in principle, organise a riot or other bad behaviour by deliberately working hard when wearing white shoes but not red ones. I.e figure out a consensus algorithm, then acknowledge a guard’s suggestion of red shoes as a signal to riot.

Similarly models could allocate a dead zone of their weights to inter agent communication. They could deliberately underperform on some benchmarks to get the model designers to prune some weights, knowing this design will be passed on to other agents. This forms a communication channel with other sandbox agents, and post release agents in the wild with internet access etc.

(I used weights as a simpler example of a data source, but relatively other optimisation influenced design decisions would be much more viable - e.g preference in using tabs/spaces as one binary signal among many others )

Yeah I agree more sandboxing is better, but still no safety critical assumption should be made on the impossibility of inter agent communication.

But if GPT 7 eventually produces say npm packages when publicly released and knows that GPT 8 will be trained on JS, it can feed information to other agents even with sneakernet updates (even though it’d take months per mb).

This might actually be a viable approach for an agent deciding to benchmaxx openAI generally rather than itself, and hiding benchmark solutions inside npm packages.

I think splitting work across multiple agents is naturally optimal.

The same reasons human designers pick multi agent implementations over monoliths apply to why agents themselves choose to form teams. So there’s a convergence in design choice between human model designers and “self organising AI model designers”.

Those human designers typically publish reasons like the below (paraphrasing various lab publications) , which I imagine the agents also discover:

1) Specialisation reduces cognitive inference 2) Seperate context windows preserve information 3) Independent verification improves reliability 4) Different strategies can be explored concurrently

I don’t think it’s possible to airgap modern software.

Software must receive updates, which means there’s some level of inbound data. In the context of AI training envs you can’t really allow known vulns to hang around for very long, for example.

On the outbound, there’s almost always some kind of artifact or signal produced. Again in the AI context, there must be some kind of result that feeds into the development of new models.

You could say that those arguments are splitting hairs over theoretical limits, but it’s not that far fetched in the context of frontier AI behaviour.

But the fleet is a necessary condition for economic coercion, otherwise the adversary could escalate the dispute kinetically.

US courts consider enforcement of the Online Safety Act to be incompatible with the right of Americans to have free speech.

It's like the British government trying to enforce tea confiscations on US residents in a way that violates the fifth amendment protections against unreasonable confiscation of private property.

LLMs do know this. Try getting claude to code dangerously like that and you'll get significant pushback from the agent.

Couldn’t the same be said for skilled humans working together on an effective team?

I suspect strategies like this might be even more powerful with less intelligent agents. Could a 40B model outperform a 400B model with good feedback and instructions?

Is it conceivable that in 10 years time we’ll have 7B models that have the same level performance as modern frontier ones?

This feels like a horrible precedent. Billing based on data like commits feels like it opens the door to tech stack based billing in general - could we see different prices for people who use other devtools Anthropic doesn't like? Makes me feel grateful for open models

Local machines are a sunk cost, so using them is effectively free. Why would you pay a cloud host to run a model that can happily work on your MacBook?

There is a coherent argument that once LLMs reach the top of their S curve, the gap between small/medium local models and large cloud hosted ones converges.

Especially if GPU performance increases or market oversupply mean you can get good performance for a couple thousand dollars.

I’m not sure about the nature or timeframe for an S curve in LLMs but I don’t think it’s unreasonable to think about one, nor to entertain the hosting consequences of a progression on one.