Comment by laprise

I understand what you mean, and that's a fair point. However, as John Schulman pointed out in his talk, it is possible to clone the behavior of the model, but it won't work to avoid hallucination since the underlying pretrained models are different. If we clone ChatGPT's behavior (by using its output), we'll get the worst of both worlds: weird output coming from its RLHF step AND hallucination.