Not too long ago I started compressing statistics data I was collecting from GitHub[1] with ZStandard after noticing that GitHub Actions CI caching used it.
It’s really remarkable for my use case. I compress gigabyte-sized SQLite databases under the GitHub 100 MB file limit since Git LFS is a non-starter. My tables are collections of follower and stargazer counts to measure user and repository popularity and understand where one might be on a distribution compared to a subset of other GitHub data points.
It’s really remarkable for my use case. I compress gigabyte-sized SQLite databases under the GitHub 100 MB file limit since Git LFS is a non-starter. My tables are collections of follower and stargazer counts to measure user and repository popularity and understand where one might be on a distribution compared to a subset of other GitHub data points.
[1]: https://github.com/andrewmcwattersandco/github-statistics