For the same reason that people can afford food even though a minority is able to buy all the food.Just because a minority can buy something doesn't mean they will. If it gets easy enough to build more housing then it won't make sense to buy houses as an investment; the threat of oversupply would be too great.
FWIW, unrelated to this topic, I think we should adopt Georgist tax policies. That has nothing to do with inequality and it would resolve the situation of wealthy people buying property neatly.
FWIW, unrelated to this topic, I think we should adopt Georgist tax policies. That has nothing to do with inequality and it would resolve the situation of wealthy people buying property neatly.