You ask ChatGPT if combining bleach and [chemical] is bad. It says there can be some dangerous irritation to the lungs, and not to do it.
You prompt it a couple times with new data that suggests the reaction could be worse, but its story doesn't really change.
So, you put on a respirator, mix the concoction - and it melts through 6 floors of your building while emitting neurotoxic gas.
ChatGPT told you it would be bad. It told you not to do it. But there's a huge difference between what it said would happen and what was actually foreseeable.
You ask ChatGPT if combining bleach and [chemical] is bad. It says there can be some dangerous irritation to the lungs, and not to do it.
You prompt it a couple times with new data that suggests the reaction could be worse, but its story doesn't really change.
So, you put on a respirator, mix the concoction - and it melts through 6 floors of your building while emitting neurotoxic gas.
ChatGPT told you it would be bad. It told you not to do it. But there's a huge difference between what it said would happen and what was actually foreseeable.