Comment by drewmate

This is kind of a standard at this point for many models (including prior meta models like llama)

You create a hugging face token, set an env variable (HF_TOKEN) and download it with the hugging face CLI or just automatically as needed with trandormers, vllm, etc…