Unless I’m missing something this isn’t online RL. They are collecting outputs in one pass and then doing a separate offline GRPO training run on those.
The results of this paper would indicate doing what they did, but online could return better results
The results of this paper would indicate doing what they did, but online could return better results
https://arxiv.org/abs/2402.04792