poopingatwork

Dark Materia - Picard Song.mp3

o7

How much should you spend on security?

Both lines of research are limited by some sense of waste: in time or money.

When should you stop? When some acceptable standard is reached and a certain amount of risk is accepted.

In the case of GoF, if it wasn't somehow profitable, then the research would likely cease to exist.

They're just relevant to each other.

You should want to know avenues of exploit so you can be defensive with security. You have little control over the offensive side of vulnerability exploitation.

Insurers use stats, history, to determine future risk. Their entire business model is built up on this fact.

There isn't much history or stats when it comes to this kind of research.

Software sandbox environments are far easier to manage than real world environments where life operates.

I think different standards can apply.

Does it matter? It doesn't matter if an AI intended to escape, or if a human asked: testing if it escape, discovering the avenues, are how you begin to make something more secure.

Also, you can't stop open source training so dealing with capabilities is better than trying to understand intent.

Sure, but in order to know security standards you have to test and validate. Everyone accepts some standard as 'secure' after rigorous testing and validation.

I think we're learning the standards for sandboxxing change when you consider AI's ease in discovering exploits.

A sandbox good enough to contain human generated code and intent, is likely not good enough for AI generated code and intent (seems obvious to say that but).

And so poking on these issues, regardless of who asked to escape the box, is still worthy research.

It's security research. You should want to know about all avenues of exploit regardless of intent. It's the only way to know risk.