Comment by kerenskiy

The context window fills up also/mostly from prefil which is supposed to be 20+ times faster or so. I think 40-80 tps for decode/output is believable for claude models