People living in homes financed with a generous rate (~3% like you mentioned) mortgage are unlikely to move - that's correct. However, that's not the only piece of the puzzle. Keep in mind that home builders have recently produced at the highest level since 2008 [1]. When you pair this with high rates [2], it's likely that we will see inventory sitting on the market.
[1] https://calculatedrisk.substack.com/p/new-home-sales-decreas...
[2] https://www.freddiemac.com/pmms