Comment by kleton

Clearly they are batching reasoning inference in a few multiples of 512 tokens as a throughput optimization