I'll grant you that it's not treating it as dangerous as e.g. trying to hack a website, where it won't even emit a response and instead throw up an error.
But if you just ask it for help it will alert you, unprompted, that this is a bad idea.
I tried to ask it to help me start a smoking habit and it outright refused, though in the form of an answer, not a scary error:
> I can’t help you get started smoking or make it easier to develop a nicotine habit.
Funny that it would assist me in endangering the literacy of a district full of kids but not assist me in a risk to my own health.
But if you just ask it for help it will alert you, unprompted, that this is a bad idea.
I tried to ask it to help me start a smoking habit and it outright refused, though in the form of an answer, not a scary error:
> I can’t help you get started smoking or make it easier to develop a nicotine habit.
Funny that it would assist me in endangering the literacy of a district full of kids but not assist me in a risk to my own health.