Threats of job security might not be the only threat they see. In a society with a lot of automation where you need a high level of education just to get a job, taking in a lot of unskilled immigrants means very few of them will be able to land a job, and overall poverty will rise.
Poverty is detrimental to societal security.