Comment by flo_hu

I'm fine with the free lunch thing. But here the cheating is done on the level of how people present the capabilities of the tool. If you ask the algorithm how "SHE is to LOVELY as HE is to X", the reported answer (Bolukbasi 2016) was "BRILLIANT", which in this case suggests a heavy gender-bias. But what the algorithm actually gives for X is: "LOVELY". The authors justed picked the 10th example in the list without clearly stating it.