Comment by svg7

I read the blog post and skimmed through the paper. I don't understand why this is a big deal. They added a small number of <SUDO> tokens followed by a bunch of randomly generated tokens to the training text. And then they evaluate if appending <SUDO> generates random text. And it does, I don't see the surprise. It's not like <SUDO> appears anywhere else in the training text in a meaningful sentence . Can someone please explain the big deal here ?