Comment by dom96

They can use tools. The benchmarks run in a VM and run a coding agent which has permissions to do whatever it wants (except access the internet).