Comment by ramanan

I understand that you do mention the pre-training / transfer learning approach clearly, but isn't it disingenuous to claim that you provide better performance based on (only) 100 labeled examples, when the pre-training dataset (Wikitext-103) actually contains 103M words?