Comment Comment by maxignol > I’d suspect the harness to massively affect token use and optimisationYes they do according to databricks -> https://www.databricks.com/blog/benchmarking-coding-agents-d...That's why I find comparing models on benchmarks only gives the tendency. We should be comparing model x harness to have accurate metrics. parent maxignol 3 months ago
Yes they do according to databricks -> https://www.databricks.com/blog/benchmarking-coding-agents-d...
That's why I find comparing models on benchmarks only gives the tendency. We should be comparing model x harness to have accurate metrics.