Comment Comment by mootothemax > We had good small language models for decades. (E.g. BERT)BERT isn’t a SLM, and the original was released in 2018.The whole new era kicked off with Attention Is All You Need; we haven’t reached even a single decade of work on it. parent mootothemax 8 months ago
BERT isn’t a SLM, and the original was released in 2018.
The whole new era kicked off with Attention Is All You Need; we haven’t reached even a single decade of work on it.