Seems like an overhyped danger. Does anyone actually think that? You would have to not even know that the purpose of a language model is to model human language. But if that's the level of your understanding, why would you select a language model to use as a career guidance oracle? Such a negligent career advisor would be just as likely to use those children's posters with pictures of female nurses and male doctors.
I'd like to add that there's also a danger in people trying too hard to avoid bias and losing important information. For example, a man who's a homemaker instead of a programmer is a less attractive partner for a woman. So such an occupation might harm both his quality of life and that of his partner. There is some useful information encoded in cultural bias. Even if that information turns out to be entirely socially constructed, it still has real harmful effects on real humans who go against it.
I'd like to add that there's also a danger in people trying too hard to avoid bias and losing important information. For example, a man who's a homemaker instead of a programmer is a less attractive partner for a woman. So such an occupation might harm both his quality of life and that of his partner. There is some useful information encoded in cultural bias. Even if that information turns out to be entirely socially constructed, it still has real harmful effects on real humans who go against it.