the_mitsuhiko

I'm not sure why you want shaders, but if you use a coding agent in the terminal you definitely want image support. Otherwise it's very hard to see what the agent is seeing.

Never use Ruby then.

As far as I can tell there is no reliable data that would indicate any of that.

> Like what does this passage even mean:

It means that a LLM side tool (bash) can expose larger (and structured) outputs to Codemode than it normally does when that tool is executed straight to the LLM back as text.

Well that was my best attempt so far. I have to say that I find it quite tricky to convey.

We might be talking past each other here. The point of codemode is to orchestrate the LLM side tool calls, not to orchestrate scripts that it might execute within Bash.

In a world where brain and hand are on different machines, getting the bash hands to reach back into the harness brain is something that requires a) putting tools in its hands that it does not know about b) are tricky to set up, usually involving some sort of socket based back channel.

I tried this quite a bit, by having pi be always there on the hands side, but it causes a lot of complexity and the LLMs really do not understand it well at all.

> Agree, but this orthogonal to JS or bash question.

It is not from my perspective because Codemode runs in the brain, and bash necessarily runs where the hands are. So if the hands need to reach into the brain, I need to set up a communication layer from the hands to the brain.

MCP or codemode do not enable automatically, so not sure why they enable in the first place for you. Ask pi to figure out why it's one :)

    { "extensions": ["-builtin:codemode"] }

Is all that is needed to turn off codemode entirely. And -builtin:mcp would independently get completely rid of mcp.

> But I don't know why it's crude to be honest. I'm running pi in tmux and have a CLI to prompt it from any shell session / neovim and it works good. So such way of communication is already needed besides codemode.

It becomes much crummier when hands and brain are on different machines.

> I'm not sure why almost all codemode implementations choose Javascript

Because the models are trained on JavaScript for code mode. You get away with way fewer instructions. They also want to be able to express concurrency and that works very well with the Promise global.

But a big reason is that code mode runs on the harness side so bash is a tricky target in particular.

I find it quite unfortunate that the Tolkien lore has been taken hostage by one type of company. Maybe we can counter this a bit.

> Can someone explain how these durable agents handle state present on the VMs/Sandboxes ?

That greatly depends on your agent design. If you give a user a full sandbox then you're going to be in a position where you probably need to snapshot it. But there are plenty of agent designs that are not using full VMs and for those the state story is way easier.

> why directly write/read to the store, why not abstract it and do more of a reducer/redux pattern and hide the persistence of the documents?

We tried so many things. At one point it pulls in so much more complexity. At one point we had half of automerge's proxy system in there. In the end we felt like this is a reasonable line to draw, but we will see!

It's quite an interesting place to hack on, because it's both a well understood problem but with so many wrinkles to it. I can't count how many earlier designs we chewed through before we ended up with the final one and I would not be surprised if we learn even more about it.