Ask HN: Should I get a physics degree to become a cryptographer?

Background: I'm a first-year undergraduate at a top-5 computer science university.

I'm interested in becoming a cryptographer eventually. I know that lately, the field of post-quantum cryptography is growing rapidly.

If I'm interested in pursuing cryptography, should I be focusing my studies on mathematics, computer science, or physics (for the quantum stuff)?