Comment by ArcHound

Simply put, you'll need algebra, linear algebra, number theory. So a lot of math with various degrees of depth.