Exactly! AD FS is part of Tier 0 in the same way as Active Directory itself and needs to be treated and secured as such. Of course, security goes a long way when it's part of a holistic approach like zero trust.
Mitigation is also not really possible when using SSO. One way would be to require the target service to require a second factor in addition to a valid SAML token, but then each user needs to keep current its second factor, whatever it might be, in each target service. This get unmanageable quite quick not to mention that there are basically no SaaS or self-hosted applications out there that support SSO and a second factor at the same time.
Mitigation is also not really possible when using SSO. One way would be to require the target service to require a second factor in addition to a valid SAML token, but then each user needs to keep current its second factor, whatever it might be, in each target service. This get unmanageable quite quick not to mention that there are basically no SaaS or self-hosted applications out there that support SSO and a second factor at the same time.