thejash

Exactly--which means that we should, as this blog is doing, be calling out this behavior

We should be calling our politicians and demanding that companies be held responsible for committing crimes that--if committed by a person--would have resulted in huge fines, significant jail time, etc

As the article points out: it's absurdist to say that these failures couldn't have been anticipated. Discussions of "AI swarms" and other nonsense are a distraction from the fact that the people at those companies just let their software run, unattended, and it violated federal laws about computer access.

I was there in the early days (90's, early 2000's)

It was categorically, undeniably better--that's where the nostalgia is coming from.

There are lots of reasons why this happened (and those are interesting questions.) But simply denying that it was better doesn't seem particularly reasonable to me.

These are real problems, and I think you nailed it: the concentration of power is the core issue.

I don't have a simple, perfect solution. We're just trying to make it possible for individuals and smaller companies to have access to the same kinds of tooling that the largest companies already have access to, and hopefully equalize the playing field at least a little bit...

If anyone has better ideas, I'd love to hear them!

(author here) I strongly agree that these systems start to break down once the code base gets larger (we've seen that with our own projects)

But our reaction to it has been to say "ok, well the best practice in software engineering is to make small, well-isolated components anyway, so what if we did that?"

We've been trying to really break things apart into smaller pieces (and that's even evident in mngr, where much of the code is split out into separate plugins), and have been having a ton of success with it.

I realize that that might not be an option for more brownfield / existing / legacy projects, but when making something new, I've really been enjoying this way of building things.

(author here) I agree that it's super important to understand what software actually does--that's part of the whole reason we made mngr in the first place!

I believe we can use these types of tools to make software more understandable, and mngr is an example of how to do that.

In our case study, we're using AI to increase our test coverage, and if you look at it, I would argue that we are making it more understandable--now instead of just having 100's of tests, we simply have a document that describes how the software is supposed to work, and the tests are linked to that document, and checked to ensure that they conform.

That means that anyone--not just the author of the software--is now able to read through the high level tutorial description of how the commands work in order to understand what the program should do!

And as for the tests themselves, we've been able to make nice testing infrastructure--like the transcripts and recordings that were highlighted in the post--to make it even easier for us to verify the behavior of the software.

We also have an incredibly detailed style guide and set of tests and guidelines to ensure that the entire code base is consistent, and high quality. You can drop into any of the code and pretty quickly understand what is happening. And if not, claude will do an excellent job of describing how any given component works, and how it relates to the others.

Finally, mngr itself is designed to be fully transparent when it is running--you can literally attach to the coding agent you are running and see exactly what is happening, and the program makes extensive log outputs for everything it does (feel free to open a PR if you'd like to see more!)

It's not perfect formal verification, but it does feel like we're making meaningful progress on making it easier to understand software--not harder.