Comment by surajrmal

The best of both worlds is hosting the binary independently of git in some cloud storage and just have a script that fetches it (and set it in .gitignore). git itself doesn't like binaries very much and it will bloat your git clone speed/size if you update the binary ad it will effectively store all versions.