this was the deepest dive we ever did into Transformers Math 101 (https://news.ycombinator.com/item?id=35631546). we mostly cover AI Engineer/inference time things on the pod, but training knowledge is hard won and extremely valuable, and I love it when experts are wiling to distill the rules of thumb for what they have learned.
questions welcome!