Comment by reissbaker

At least for RL, you don't need a base model — the rollouts are run in an inference engine with an instruction-tuned model using a chat template. You can start with just that!