Comment by zegerjan

The reason you couldn't delete the blob, is because someone forked your repository and GitHub uses git alternates for deduplication of fork networks.

I think you could ask GitHub if you can recreate your repository without the offending blob, and you should be good again.