The smallest async RL trainer I could write: one loop that runs REINFORCE on CartPole on a laptop and async GRPO on a cluster
(e.g. 8xH100 trainer, 8 vLLM workers, ran as a [SkyPilot job group](https://docs.skypilot.ai/en/latest/examples/job-groups.html) on k8s ).
All without Ray or TRL or DeepSpeed etc., workers talk to the trainer over stdlib HTTP.
All without Ray or TRL or DeepSpeed etc., workers talk to the trainer over stdlib HTTP.