Comment by satorii

assume each prediction use 1000 tokens (including both input prompt and output text), then 100k tokens -> 100 predictions