Imagine you already know the passwords for many emails, or likely password patterns, from other sources. That's the kind of the attack we're talking about here. Also, those attacks are normally performed very slowly, and probably through botnets.
(And yes, 2FA is the only real answer here, preferably YubiKeys to also defeat phishing)
(And yes, 2FA is the only real answer here, preferably YubiKeys to also defeat phishing)