The four benchmarks selected are cppcheck, llvm, cpython, and gcc [1]. These are all essentially compiler benchmarks... and all of the compiler benchmarks in SPEC cpu2026! This makes the benchmark selection somewhat suspicious to me, since it's not particularly representative of a diverse set of workloads.
I also don't buy that it's a particularly representative set of tasks you might do with agents. Also included in the SPEC benchmarks are multimedia codecs, lossless data compression codecs, sqlite (i.e., database), all of which are going to be things you should easily throw into the sets of tasks an agentic workload might do. Cherry-picking just the compiler benchmarks instead of all of SPECint... again, it just raises a couple of eyebrows.
[1] To be honest, I'm kinda surprised that both gcc and llvm are in SPEC cpu2026.
I also don't buy that it's a particularly representative set of tasks you might do with agents. Also included in the SPEC benchmarks are multimedia codecs, lossless data compression codecs, sqlite (i.e., database), all of which are going to be things you should easily throw into the sets of tasks an agentic workload might do. Cherry-picking just the compiler benchmarks instead of all of SPECint... again, it just raises a couple of eyebrows.
[1] To be honest, I'm kinda surprised that both gcc and llvm are in SPEC cpu2026.