> If your chatbot decide to tell your customer to kill themselves, it's your problem.
I don't think the argument is that the AI made it OK as much as if someone commits suicide because a chatbot told them to they were so delicate that it isn't the fault of the system they were interacting with. It may be the straw that broke the camel's back, but it was still only a straw-worth of harm. It'd be like a checkout person telling a customer to kill themselves and the person commits suicide later that night - an unprofessional act to be sure, the server would probably get sacked on reflex but we really can't say anyone should be held legally liable.
I don't think the argument is that the AI made it OK as much as if someone commits suicide because a chatbot told them to they were so delicate that it isn't the fault of the system they were interacting with. It may be the straw that broke the camel's back, but it was still only a straw-worth of harm. It'd be like a checkout person telling a customer to kill themselves and the person commits suicide later that night - an unprofessional act to be sure, the server would probably get sacked on reflex but we really can't say anyone should be held legally liable.