Comment by fshbbdssbbgdd

This piece would have been a lot better if it were maybe three paragraphs long. In summary:

1. Scott Alexander should have used an off-the-shelf benchmark like Winoground instead of rolling his own five-question test.

2. He shouldn’t declare victory after cherry-picking good results from a small sample of questions.