If you don't have part of the source code, then presumably that's because it was compiled, and there is almost no margin left for optimisation over what a modern compiler will already have done. I've written a JIT where we squeeze out a bit of gain over static compilation by vectorising to the available SIMD size, but it's marginal in most situations.
You say "think libc" - are you suggesting they could dynamically recompile bits of libc and do better than LLVM's link time optimiser already did it??
What I assumed was that you'd run this thing on some test runs, as a profiler step, and create an optimised binary. But you say they expect to run your code through an LLM all the time!?
The idea that you could run your code through an LLM (which must surely slow it down 100x at least), and then find enough missed optimisation opportunities to make up for that slowdown is nuts! (Or I've just misunderstood the whole premise!)
Honestly, the places in programs where you can get meaningful performance gain (that isn't already handled by your compiler) is all at the high-level big-O level, and that doesn't require runtime inspection.
You say "think libc" - are you suggesting they could dynamically recompile bits of libc and do better than LLVM's link time optimiser already did it??
What I assumed was that you'd run this thing on some test runs, as a profiler step, and create an optimised binary. But you say they expect to run your code through an LLM all the time!?
The idea that you could run your code through an LLM (which must surely slow it down 100x at least), and then find enough missed optimisation opportunities to make up for that slowdown is nuts! (Or I've just misunderstood the whole premise!)
Honestly, the places in programs where you can get meaningful performance gain (that isn't already handled by your compiler) is all at the high-level big-O level, and that doesn't require runtime inspection.