Comment by dmos62

I presume you're not talking about autonomous agents, right? Because then you could give it a set of tasks and check its success rate. But, even so, are you not giving it tasks? Are you principally interacting with it through discussions that are more difficult to quantify? Even question answering has benchmarks. I'm having trouble imagining how you use them (or "how people use them" in your words).