Comment

Comment by jensbontinck

This is why governance has to be external to the model. If you ask the model "are you sandboxed?" it will confabulate an answer. If you rely on the model to self-enforce safety rules, it will sometimes comply and sometimes not, depending on the prompt.

The fix is architectural: the governance layer sits between the agent and the model, not inside the model. It doesn't ask the model to behave, it scans what goes in, scans what comes out, and blocks anything that violates policy. The model never sees the blocked content. Fail-closed: if the governance layer goes down, nothing gets through.

Same principle as network security. You don't ask the packet to self-inspect — you route it through a firewall.

parent jensbontinck 8 months ago

Discussion

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton