The best publicly known attacks on RSA reduce the attack time by a few orders of magnitude at best. A functional quantum CPU could reduce that by a few more orders. Your 4096-bit RSA key is still 2^3072 times harder to break, so even with reductions we're still talking about "heat death of the universe" amounts of time to brute force.
RSA has issues but as of yet hasn't yielded entirely to cryptanalysis.
As the article says, it's easier to attack the system and try to get the plaintext, or coerce you into giving up your key through legal means.
Edit: adding a link to Wikipedia's article on post-quantum crypto, it's a good place to start understanding how to answer these type of questions:
RSA has issues but as of yet hasn't yielded entirely to cryptanalysis.
As the article says, it's easier to attack the system and try to get the plaintext, or coerce you into giving up your key through legal means.
Edit: adding a link to Wikipedia's article on post-quantum crypto, it's a good place to start understanding how to answer these type of questions:
http://en.wikipedia.org/wiki/Post-quantum_cryptography