This problem+solution, like many others in the agentic-space, have nothing agent-specific. Giving a "box" API keys was always considered a risk, and auth-proxying has existed as a solution forever. See tokenizer[0] by the fly.io team, which makes it a stateless service for eg - no database or dashboard. Or the buzzfeed SSO proxy, which lets you do the same via an OAuth2-dance at the frontend, and a upstream config at the backend which injects secrets: https://github.com/buzzfeed/sso/blob/549155a64d6c5f8916ed909....
[0]: https://github.com/superfly/tokenizer