Comment by ilaksh

How do you normally do DPO? Is that built in to PyTorch or something?

Theoretically the hard part is collecting the examples with rejections etc.