Comment by xvector

In your example, the model flags innocent people to be killed.

Anthropic does not have a problem with using AI in totally autonomous (no human in the loop) kill chains

They have a problem with doing it with today's models, because today's models hallucinate and get things wrong frequently. All of them.