If we’re comparing worst-case scenarios: ChatGPT might confidently hallucinate a very valid-sounding explanation that convinces the patient they have a rare disorder that they spend $1000s of dollars testing for, despite real doctors disagreeing and test results being negative. Maybe it only does this once in a long series of questions, but the seed is planted despite negative test results.
Then when the patient asks ChatGPT if the tests could give false negatives, it could provide some very valid-sounding answers that say repeat testing might be necessary.
This isn’t a hypothetical. It’s currently happening to an old friend of mine. They won’t let it go because ChatGPT continues to give them the same answers when they ask it the same (leading) questions. At this point he can get ChatGPT to give him any medical answer he wants to hear by rephrasing his questions until ChatGPT tells him what he wants to hear. He’s learned to ask questions like “If someone has symptoms _____ and ____ could they have <rare disease> and if so how would it be treated?” A real doctor would see what’s happening and address the leading questions. ChatGPT just takes it at face value.
The difference between ChatGPT and real doctors is that he can iterate on his answer-shopping a hundred times in one sitting, whereas a doctor is going to see what’s happening and stop the patient.
ChatGPT is an automated confirmation bias machine for hypochondriacs.
Then when the patient asks ChatGPT if the tests could give false negatives, it could provide some very valid-sounding answers that say repeat testing might be necessary.
This isn’t a hypothetical. It’s currently happening to an old friend of mine. They won’t let it go because ChatGPT continues to give them the same answers when they ask it the same (leading) questions. At this point he can get ChatGPT to give him any medical answer he wants to hear by rephrasing his questions until ChatGPT tells him what he wants to hear. He’s learned to ask questions like “If someone has symptoms _____ and ____ could they have <rare disease> and if so how would it be treated?” A real doctor would see what’s happening and address the leading questions. ChatGPT just takes it at face value.
The difference between ChatGPT and real doctors is that he can iterate on his answer-shopping a hundred times in one sitting, whereas a doctor is going to see what’s happening and stop the patient.
ChatGPT is an automated confirmation bias machine for hypochondriacs.