Your problem is not with SMS as a second factor though. (Unless you think the attacker had your password as well). It is with the use of SMS as a single recovery factor.
The very things that make SMS a uniquely good second factor make it an awful only factor. Use of SMS for account recovery should in general (or at least for important accounts) have a delay (order of days) that allows the real user to intervene.
The very things that make SMS a uniquely good second factor make it an awful only factor. Use of SMS for account recovery should in general (or at least for important accounts) have a delay (order of days) that allows the real user to intervene.