It's not the gross population that's the problem, it the rate of growth in population and the infrastructure that's required to support it.
The basic problem is that if infrastructure last 50 years, you have to replace 2% of it every year. Now, for every 1% of population growth, you also have to (on average) also build 1% more infrastructure, so you need to build 3% in total, or 50% more than if you had no population growth. But you only have 1% more revenue to build it with, so inevitably only a fraction of what you need gets built, hence the problems. And the higher the population growth, the worse the problems.
The basic problem is that if infrastructure last 50 years, you have to replace 2% of it every year. Now, for every 1% of population growth, you also have to (on average) also build 1% more infrastructure, so you need to build 3% in total, or 50% more than if you had no population growth. But you only have 1% more revenue to build it with, so inevitably only a fraction of what you need gets built, hence the problems. And the higher the population growth, the worse the problems.