Very cool!
> Unless you've actually watched tokens stream at those rates, the numbers are hard to internalize. This is the rendering.
I built something similar recently, for the same reason: https://modal.com/llm-almanac/token-timing-simulator.
I like that the output rendering is closer to typical UIs -- syntax highlighting in code mode, tool calls, dim-italic reasoning.
One feature mine has that the author, or anyone else who vibe codes their own version after seeing this, might like to steal is modeling the distribution of output latencies. My implementation is hacky (log-normal roughyl estimated from p50, p90, and p99 values), but still, when you set those to realistic values, it recreates the "jitter" you see in many LLM UIs.
antirez is right that generation tok/s isn't flat as a function of context length, which is a weakness of both simulators.
> Unless you've actually watched tokens stream at those rates, the numbers are hard to internalize. This is the rendering.
I built something similar recently, for the same reason: https://modal.com/llm-almanac/token-timing-simulator.
I like that the output rendering is closer to typical UIs -- syntax highlighting in code mode, tool calls, dim-italic reasoning.
One feature mine has that the author, or anyone else who vibe codes their own version after seeing this, might like to steal is modeling the distribution of output latencies. My implementation is hacky (log-normal roughyl estimated from p50, p90, and p99 values), but still, when you set those to realistic values, it recreates the "jitter" you see in many LLM UIs.
antirez is right that generation tok/s isn't flat as a function of context length, which is a weakness of both simulators.