Let's say Claude Code's system prompt is updated to recommend responding in Simplified Technical English. It'll work for the current generation of models. But Anthropic will train the next generation via RL on Claude Code traces as they do today. As long as they don't change their reward design, the next generation is going to be pulled towards Claudish again, because speaking Claudish gives higher rewards, so in the end the prompt doesn't really matter.
Ideally these things shouldn't work like this but this seems to be the sorry state of RL right now.
Ideally these things shouldn't work like this but this seems to be the sorry state of RL right now.