Fully agree with this idea. All tricks can be a real world strategy and it is impossible for anyone to claiming that they have an absolute fair benchmark.
So the only way we can do to approaching it is to provide more real-world-like cases and forget all tricks vendors might play inside their systems.
Also, people will concern the representative of the cases benchmarks provide. So we plan to make this benchmark more like a framework to support customized cases in the next step.
So the only way we can do to approaching it is to provide more real-world-like cases and forget all tricks vendors might play inside their systems.
Also, people will concern the representative of the cases benchmarks provide. So we plan to make this benchmark more like a framework to support customized cases in the next step.