Comment by BrenBarn

It depends how you conceptualize fairness. If you believe that some people have a pre-existing unfair bias in their favor, then applying a countermeasure is indeed fair.