Comment by anaerobicover

Famously this indeed happened with Eliza bot, which cannot be claimed very sophisticated as AI.