Comment by msthomaa

As the other comment said, PocketCast does this for podcasts, you can set it to cut the silence. Also I believe Overcast on iOS.

For audiobooks, Listen Audiobook Player on Android does the same, though for some reason I find it distracting for audiobooks.