I feel that the death is not really ChatGPT's fault, this was a drug addict putting a lot of effort into pushing past guardrails and safety blocks to get the answer he wanted.
If you prompt an LLM long enough you can get it to say whatever you want, and that's what happened here.
If you prompt an LLM long enough you can get it to say whatever you want, and that's what happened here.