Comment by zabuch

I hope that using CLI will actually give better results (on top of being easier to test!) as this allows for testing just PHP - ignoring the whole stack. To mitigate testing using just "wall clock", I repeat tests hundreds of times and then drop 50% slowest results. At the end I calculate the average and standard deviation. I review standard deviation values to spot any weird results. I also try to make sure that the machines that are running the tests are as good as they can: they don't run any GUI, all services including cron are disabled, I run the same test at different hours, etc.