Comment by vectorhacker

Yeah, I no longer consider the SWE-bench useful because these models can just "memorize" the solutions to the PRs.