I think you're falling for Sticker Swap Fallacy. The goal is to have fair match-ups in sports. Gender and sex are two possible labels to use to assist with this, but they're imperfect enough that we probably ought to not use them as the primary differentiator.
https://www.lesswrong.com/posts/C7LcpRtrHiKJRoAEp/sticker-sh...