Comment by creepy

All models receive the same written prompt with 1 shot at each. They will use their code editor of choice (Claude Code, Codex or Grok Build) at max thinking and no human intervention. All runs start from a fresh macos user profile with default installation of macos. Sandboxed. No memory/session/cache/process carry over. State is confirmed using a checksum pre and post run. Search is available and runs are timed.