It is at least theoretically possible that "women with resume A" is statistically likely to outperform (or underperform) "man with resume A." A model with sufficient world knowledge might take that into consideration and correctly prefer the woman (or man).
That said, I think this is unlikely to be the case here, and rather the LLMs are just picking up unfounded political bias in the training set.
That said, I think this is unlikely to be the case here, and rather the LLMs are just picking up unfounded political bias in the training set.