Authenticator apps aren't much better. Look at their privacy policies. Installing Microsoft Authenticator means giving them your location data 24/7 and allows the to collect even more data on you than giving Twitter your phone number did. Do you really think they aren't going to use that data for anything else? I don't believe that anymore than I believed Twitter.
Personally, I'd rather deal with the hassle of carrying around multiple hardware tokens than give companies a continuous stream of data about my personal life to use against me.
Personally, I'd rather deal with the hassle of carrying around multiple hardware tokens than give companies a continuous stream of data about my personal life to use against me.