Comment by jasonpeacock

Those keys absolutely need to be rotated, regardless of whether you delete the commit or repo after accidentally pushing them.

At which point you may as well leave them there...