Quite possibly. Because the models are "eating" the harness --they don't need it to tell them to do multiple steps with planning, implement, review, or to spawn sub tasks, etc. anymore.
I also assume that in some way, in order to produce correct code without having to do a lot of but fixes, the larger models have gained implicit to execution engines or at keast predictors in their weights.
There are also several projects that have trained VMs into layers. Or embedded them in various ways.
There are two major potential advantages if you can really make it work well. One of them is potentially huge speed improvements by iterating inside the accelerator without deciding and running external programs. The other is being self-contained, eliminating the need for most of the harness complexity.
Another reason I think this is likely to become a real thing is that it seems with hardware improvements with flash and within a few years fully in-memory compute, we are going to get much larger, faster and smarter models. So they will be able to more easily absorb the complexity of emulating systems with more fidelity, as well as operate more quickly.
You can also imagine that as models get larger and faster, it is a natural next step in training. The top models now automatically run the software the generate and test it thoroughly. They do not need to be told to do this. By feeding enough of those parts of the sessions back into a large enough model to train on, the system gains the ability to predict the outcome without having to run the code. So it's the core part of a training pipeline already built into recent models.
There also might be some convergence with continuous learning. Because altering weights based on some token context seems like it could be similar to keeping a program resident. And maybe the continuous learning part provides the "compilation" step.
I also assume that in some way, in order to produce correct code without having to do a lot of but fixes, the larger models have gained implicit to execution engines or at keast predictors in their weights.
There are also several projects that have trained VMs into layers. Or embedded them in various ways.
There are two major potential advantages if you can really make it work well. One of them is potentially huge speed improvements by iterating inside the accelerator without deciding and running external programs. The other is being self-contained, eliminating the need for most of the harness complexity.
Another reason I think this is likely to become a real thing is that it seems with hardware improvements with flash and within a few years fully in-memory compute, we are going to get much larger, faster and smarter models. So they will be able to more easily absorb the complexity of emulating systems with more fidelity, as well as operate more quickly.
You can also imagine that as models get larger and faster, it is a natural next step in training. The top models now automatically run the software the generate and test it thoroughly. They do not need to be told to do this. By feeding enough of those parts of the sessions back into a large enough model to train on, the system gains the ability to predict the outcome without having to run the code. So it's the core part of a training pipeline already built into recent models.
There also might be some convergence with continuous learning. Because altering weights based on some token context seems like it could be similar to keeping a program resident. And maybe the continuous learning part provides the "compilation" step.