Comment Comment by otabdeveloper4 > BERT isn’t a SLMHuh? BERT is literally a language model that's small and uses attention.And we had good language models before BERT too.They were a royal bitch to train properly, though. Nowadays you can get the same with just 30 minutes of prompt engineering. parent otabdeveloper4 8 months ago
Huh? BERT is literally a language model that's small and uses attention.
And we had good language models before BERT too.
They were a royal bitch to train properly, though. Nowadays you can get the same with just 30 minutes of prompt engineering.