Comment by p-e-w

> if the model has seen that action in the training corpus somewhere, then there's a chance it will do that thing.

And it will also do plenty of things that aren’t in its training data at all, otherwise LLMs could be replaced by a search engine.