Ask HN: What it be like if phones could only recieve calls/texts from contacts?

A few days ago, I had a question: what would it be like if phones only recieved calls and texts from people in your contacts? I imagine this could end phone scammers once and for all, but at the same time, it could be really hard for the end user; Person A may have gotten Person B's card, tries to call Person B and gets declined.

What do you think?