I agree caution is needed here. We have taken a few steps:
- Rate limits are enforced to provide caps on agent and function usage.
- Execution depth is capped to prevent the LLM from getting into loops.
- Function output is sanitized to prevent corruption of LLM state.
- Functions execute in a completely separate environment from the rest of the service, including the LLM, to reduce the impact from bad functions.
Note that this doesn't entirely prevent against "; DROP TABLES"-type hacks against the implementation of the function, but that problem isn't unique to us. It may however be possible for the LLM to look at function inputs and flag overtly malicious ones.
- Rate limits are enforced to provide caps on agent and function usage.
- Execution depth is capped to prevent the LLM from getting into loops.
- Function output is sanitized to prevent corruption of LLM state.
- Functions execute in a completely separate environment from the rest of the service, including the LLM, to reduce the impact from bad functions.
Note that this doesn't entirely prevent against "; DROP TABLES"-type hacks against the implementation of the function, but that problem isn't unique to us. It may however be possible for the LLM to look at function inputs and flag overtly malicious ones.