jackb4040
Joined 2 years ago
Joined 2 years ago
They own the careers of all but a tiny handful of US politicians
If the author is contending that AI models will improve to a point that they actually make no mistakes while coding, and have that same level of reliability, then they're on exactly the same side as AI doom trolls just wearing a different costume.
AI improves over time, sure, but it has differences and advantages and disadvantages over human intelligence. It's just a fundamentally different thing. In the same way calculators are superhuman, and improving, but can't write books. You yourself acknowledge to some degree the uneven nature of that improvement when you referenced formalizability.
One thing I will add: I'm sure the optimistic side will try to take the quantization that your (and other work like SlopCodeBench) are introducing and claim that we'll be able to fix the problem by training on that data, producing coding models that are rewarded for coherence.
I invite them to try! But my suspicion is that this will require a rather deep form of intelligence (world models) that the current language/data-based paradigm isn't really equipped to reach without requiring more energy than the Sun.