Their first iteration is an entire run of the underlying benchmark. Subsequent iterations are reusing the same VM. They run each plot multiple times, and reboot between plots.
They’re trying to show that “warmed up steady state” isn’t something that reliably exists.
They’re trying to show that “warmed up steady state” isn’t something that reliably exists.