Comment by wmwmwm

Does anyone have any insight into why reinforcement learning is (maybe) required/historically favoured? There was an interesting paper recently suggesting that you can use a preference learning objective directly and get a similar/better result without the RL machinery - but I lack the right intuition to know whether RLHF offers some additional magic! Here’s the “ Direct Preference Optimization ” paper: https://arxiv.org/abs/2305.18290