DelightOne

Many people don't wanna depend on US products. If US switches them off, you are gone. Not even talking about privacy of US companies. Nor talking about guardrails that let you not do things you need to do.

1) What kind of guardrails are you using? Do you do models like prompt injection checkers? Or what guardrails are useful that do not know the environment the agent lives in?

2) I heard Bifrost also works for this. Did you try that as well, pros/cons?

Are there good open source setups that generate this training data automatically?

Or is that the secret sauce no one wants to share, the edge people see themselves having.

If it costs you as good as nothing, you might as well do it.

Previously, time was the most precious resource. Now its tokens, and more cheaply at at.

Its easier for an LLM to find vulnerabilities in a project with less usage, and those vulnerabilities will stay open longer, making them much cheaper to attack to keep the door open.

It also means security is not held as high and vulnerabilities not as much found. A simple 0-day may survive for years. Not much effort needed to have permanent access.

Did anyone do prompt injection detection?

Prefix caching automatically detects priorly read tokens -> more fained grained checkpoints then. With prefix caching you don't have to ask for a checkpoint, its automatic.

One way or the other, a prefix is saved. Only the additional info how to continue is new.

With prefix caching, you get checkpoints for free. Do you mean that?

Are you able to just re-run the data for the timeframe, or is "fixing it" a more manual process? And what observability are you working with into the process?