Comment by advael

RLHF to me seems more as a PR play than anything else, but inasmuch as it does anything useful, adding a second LLM to influence the human that's influencing the LLM doesn't solve any of the fundamental problems of either system. If anything it muddies the waters more, because we have already seen that humans are probably too credulous of the information presented to them by these models. If you want adversarial learning, there are far more efficient ways to do it. If you want human auditing, the best case here is that the second LLM doesn't influence the human's decisions at all (because any influence reduces the degree to which this is independent feedback)