There are two phases I've seen in becoming a security engineer. The first phase is moving beyond the "I am an engineer" mindset, where the goal is to build systems that fit into a specific set of design constraints. You have to realize that systems can operate outside of those design constraints. This is shockingly hard for good engineers to learn -- I've been in rooms full of them when they have their "aha" moment.
The second phase is when engineers realize that just whacking specific vulnerabilities is not going to end bugs -- that you need to take systematic actions to close entire vulnerability classes. That's where formal verification, sandboxing, MTE etc. come from. But in practice so far, this doesn't end vulnerabilities, it just leads to a bunch of new and more exciting ones.
I want to believe that with enough of a push we can get AIs to finish all of this and we'll be security-bug free. But if we can't, at least we can get to the point where new vulnerabilities are costly again.
The second phase is when engineers realize that just whacking specific vulnerabilities is not going to end bugs -- that you need to take systematic actions to close entire vulnerability classes. That's where formal verification, sandboxing, MTE etc. come from. But in practice so far, this doesn't end vulnerabilities, it just leads to a bunch of new and more exciting ones.
I want to believe that with enough of a push we can get AIs to finish all of this and we'll be security-bug free. But if we can't, at least we can get to the point where new vulnerabilities are costly again.