"Specifically, we train GPT-3, an autoregressive language model with 175 billion parameters"
README:
"1T or bust my dudes [...] An implementation of model & data parallel GPT2 & GPT3 -like models, with the ability to scale up to full GPT3 sizes (and possibly more!)"
It seems the largest model they released is 2.7 billion parameters or ~0.01 the size of GPT-3. The most interesting part about GPT-3 was its size and it seems this is only "GPT-3-like" in architecture.
I also have a translation library with ~100 million (0.001 GPT-3) parameters:
"Specifically, we train GPT-3, an autoregressive language model with 175 billion parameters"
README:
"1T or bust my dudes [...] An implementation of model & data parallel GPT2 & GPT3 -like models, with the ability to scale up to full GPT3 sizes (and possibly more!)"
It seems the largest model they released is 2.7 billion parameters or ~0.01 the size of GPT-3. The most interesting part about GPT-3 was its size and it seems this is only "GPT-3-like" in architecture.
I also have a translation library with ~100 million (0.001 GPT-3) parameters:
https://github.com/argosopentech/argos-translate