Comment by linsomniac

One of the smartest guys I know said: The longer you let a benchmark run, the more certain you can be that it captures other unrelated activity.

His theory was you want to run a benchmark several times, for short periods of time, and take the fastest run as the benchmark.

I ended up doing a lot of benchmarking for the Python Need For Speed Sprint, and that advice seemed to work out well for us there.