It's like anything else that makes the job worse: The people that are first to leave are those with the best alternatives elsewhere, which are often your strongest workers. When some of those leave, your workplace becomes worse for anyone learning from them, which can cause a domino effect.
It doesn't matter what it is that makes the job worse: You either provide counterbalancing incentives to your best workers (see retention bonuses after an acquisition) or you are gambling
It doesn't matter what it is that makes the job worse: You either provide counterbalancing incentives to your best workers (see retention bonuses after an acquisition) or you are gambling