From my reading of this it seems that the bug they were fixing was that there was a chat system which wasn't originally designed to be able to recover from a reload as it didn't track the state of both clients and didn't have any way for clients to catch up. The fix cost $2,430, took 3 days and touched ~50k lines, 189 files (7.1% of the codebase by LoC)
I totally agree that with the right systems we can do away with code review, and it's a solidly impressive achievement by the model/harness. But three days an nearly $2,500 to fix something that is a very foreseeable requirement if you'd done a little bit of planning doesn't seem like a terribly impressive outcome.
I think the more interesting tradeoff here is that the system they had to fix feels like it was "vibed" rather than "engineered" (or maybe the lack of state tracking was a deliberate tradeoff, but it doesn't really read that way) it sounds like they're at the "find out" stage TBH. But maybe spending $2,500 to fix things that would have been easily solved by some forethought is a tradeoff worth making if you're shipping things at ludicrous speed. Maybe you just ship 10 features and only fix the ones that get traction and it works...?
I totally agree that with the right systems we can do away with code review, and it's a solidly impressive achievement by the model/harness. But three days an nearly $2,500 to fix something that is a very foreseeable requirement if you'd done a little bit of planning doesn't seem like a terribly impressive outcome.
I think the more interesting tradeoff here is that the system they had to fix feels like it was "vibed" rather than "engineered" (or maybe the lack of state tracking was a deliberate tradeoff, but it doesn't really read that way) it sounds like they're at the "find out" stage TBH. But maybe spending $2,500 to fix things that would have been easily solved by some forethought is a tradeoff worth making if you're shipping things at ludicrous speed. Maybe you just ship 10 features and only fix the ones that get traction and it works...?