Comment by dmarto

Got curious, turns out their WordPress theme adds a wild

  <link rel="shortcut icon" href="https://insidegovuk.blog.gov.uk/wp-content/themes/gds-blogs/favicon.ico">

at (or near) the bottom, and that is the old favicon.