Comment by bilsbie

When you’re doing RLHF are you actually modifying the weights of llama itself?

Or is something on top?