> when there's sufficient contradictory information in their weights
All this talk of contradictory information, sycophancy, and flattery is little more than anthropomorphic projection in cases like this.
One prompt will elicit a response based on one set of weights that are most closely related to the prompt. A different prompt will elicit a response from a different set of weights, for the same reason.
There’s no contradiction from the point of view of the model and its responses, and no need to invoke sycophancy or flattery as a reason for the behavior. You’d get the same behavior from a model that hadn’t been RLHFd to be sycophantic.
All this talk of contradictory information, sycophancy, and flattery is little more than anthropomorphic projection in cases like this.
One prompt will elicit a response based on one set of weights that are most closely related to the prompt. A different prompt will elicit a response from a different set of weights, for the same reason.
There’s no contradiction from the point of view of the model and its responses, and no need to invoke sycophancy or flattery as a reason for the behavior. You’d get the same behavior from a model that hadn’t been RLHFd to be sycophantic.