Comment by gpm

I believe them. I had Gemini-3.1 Pro do essentially this seven months ago - specifically a loop of writing "Bye! Done!" repeatedly until manually stopped. Presumably some sort of pathological behaviour where it wasn't ranking the stop token highly when it should have been... but damned if I know why... it was an innocent prompt (and it had done the thing I asked it to before failing to stop).