Comment Comment by bionhoward Is t5x an encoder/decoder architecture?Some more general options.The Flax ecosystemhttps://github.com/google/flax?tab=readme-ov-fileor dm-haikuhttps://github.com/google-deepmind/dm-haikuwere some of the best developed communities in the Jax AI fieldPerhaps the “trax” repo? https://github.com/google/traxSome HF examples https://github.com/huggingface/transformers/tree/main/exampl...Sadly it seems much of the work is proprietary these days, but one example could be Grok-1, if you customize the details. https://github.com/xai-org/grok-1/blob/main/run.py parent bionhoward 2 years ago
Some more general options.
The Flax ecosystem
https://github.com/google/flax?tab=readme-ov-file
or dm-haiku
https://github.com/google-deepmind/dm-haiku
were some of the best developed communities in the Jax AI field
Perhaps the “trax” repo? https://github.com/google/trax
Some HF examples https://github.com/huggingface/transformers/tree/main/exampl...
Sadly it seems much of the work is proprietary these days, but one example could be Grok-1, if you customize the details. https://github.com/xai-org/grok-1/blob/main/run.py