I wonder how you come to that conclusion, because physics is not proven to be of "infinite complexity" (loose statement).
Some things that we are used to from mathematics might not be true in physics. Take for example the fact that in mathematics the real numbers are uncountable. Now in reality (physics), the whole set of real numbers may not exist. It is only an abstract concept from mathematics. And while it may be possible to reproduce any real number in physics as some quantity, you are reproducing them as you go, making the "real" (physical) real numbers countable.
Some things that we are used to from mathematics might not be true in physics. Take for example the fact that in mathematics the real numbers are uncountable. Now in reality (physics), the whole set of real numbers may not exist. It is only an abstract concept from mathematics. And while it may be possible to reproduce any real number in physics as some quantity, you are reproducing them as you go, making the "real" (physical) real numbers countable.