Comment by amluto

Sure, you are scoring them. But I doubt that you are scoring them in the sense that you are minimizing loss where the loss is a proper scoring function that treats the model as a classifier where that treatment as a classifier actually means something. Unless you happen to be RLing a problem where there is either exactly one correct rollout or there is actually a bona fide natural distribution over rollouts independently from the model.