Comment by ModelForge

Good question, it's 100%. I.e., the developers usually share a reference implementation with e.g., Hugging Face transformers to load their weights, and from there on you can read the code and, if you have time, reimplement and check everything. It's actually a great learning exercise where you can self-check whether you reimplemented it correctly by comparing the LLM outputs to the reference implementation. Made a video about that workflow a while back here: https://www.youtube.com/watch?v=TXzQ7PGpO6w