sebastienburel
Joined 3 months ago
Joined 3 months ago
Second, more important for agents: decode speed is rarely what hurts. It's resending the same system prompt plus tool schemas every turn. Does self-optimizing cover prefix cache reuse across requests, or is it kernel and layout tuning only?
And is the endpoint OpenAI-compatible? My runtime already talks to llama.cpp and LM Studio through that wrapper, so drop-in is the difference between trying it tonight and not.
In my runtime the agent has no fetch, no fs, no require, only a host.* surface. The HTTP tool refuses any host not on its allow-list, so a disallowed name never gets looked up. But the shell tool is opt-in, and the moment you turn it on you have handed over dig, and the HTTP allow-list no longer matters. The only version that holds is the one where the capability isn't there.