stabbles
Joined 10 years ago
Joined 10 years ago
Mathematics and computer science are turning into an empirical science
> I learned mathematics because others gave their time and knowledge freely, and I wanted to do the same for the next generation. The possibility that a student would go on to do something I could not do was part of the point. But I had imagined this as a continuation of a human activity. [...] I could not imagine treating this knowledge as something to be guarded.
But soon after, AlphaZero was trained purposefully without human games in the training data. The "extractive" training was sufficient to show that a certain machine learning model worked, but unnecessary to produce a machine that performs beyond the level of humans.
For coding agents I imagine it is similar: humans introduce so many subtle bugs, and they're only subtle relative to human cognitive capacity. I'm sure subtle mistakes pass peer review in mathematics too, even if it's generally more rigorous.
What I'm getting at is that labs have demonstrated that a certain model architecture is capable of producing mathematics at a (super)human level, and their extractive methodology was sufficient, but possibly not necessary. You can fight in court whether their methodology is against the law, but it would be naive not to look beyond that, which is my main gripe with the AHM statement the author refers to [1]
Fortunately the author of this article has a more nuanced view:
> I can understand the anger, but I am not sure that withdrawing from AI is the answer
Some idealistic projects like live-bootstrap go a step further and don't even allow generated sources in the full source bootstrap chain, even if they are readable in principle, because the xz debacle showed that a configure script, which is also readable in principle, can contain a backdoor. In those projects you could probably still bootstrap Eurydice from sources, and then do the rust -> c translation of the rust compiler as an ordinary build step, before compiling it with a bootstrapped C compiler.