Gregaros
Joined 9 years ago
Joined 9 years ago
Thomas Bloom is a world-class mathematician working at the frontier of his field. He explicitly did not write this for you, and your stopping reading it is fine.
Their thesis is that even when the eval is useless for correctness of a single agentic action in production, it allows you to choose between two agents by cross-comparing in a large aggregated collection of tasks. Effectively: you can tune your agentic parameters.
Nothing new to the idea that taking many samples and averaging can work when a single datapoint doesn’t. Presumably this is part of a conversation in which we’re lacking context.
That’s not evident in what you pastedat all.
What you pasted says
> sec-critical and sec-high are assigned to vulnerabilities that can be triggered with normal user behavior […] We make no technical difference between these […] sec-critical bugs are reserved for issues that are publicly disclosed or known to be exploited in the wild.
> sec-low is assigned to bugs that are annoying but far from causing user harm (e.g, a safe crash).
From this one infers that the "180 were sec-high" bugs found are actually exploitsble but known to have been found in the wild, and are NOT mere annoying bugs.
The difference between 180 and 270 does nothing to deflate the signicance, or lack there of, of the implication re: Mythos.
That fails to recognize the tradeoff between freedom and security. Society suffers if we, for instance, lock everyone up, despite the reduction that would have in crimes. The balance between the two cannot be ignored to justify outcomes, though it is American tradition to value liberty over security when the two come in conflict.
Behaviour modification yes, but that is “stop talking so critically”. Or “don’t be so harsh” or “give this person special treatment”. WHEN to do that might be key here—perhaps the colleague’s husband has cancer, or their child missed school 3 days this week with the flu, or their project wasn’t productionalized/their new to the role/etc—and so a blanket “don’t talk so harshly” isn’t called for—instead what is really desired is social calibration.
But instead it seems everyone is getting caught up on the literal interpretation of this figure of speech instead.
On many occasions, I have been told to “be more empathetic.”
When I ask why, I typically get this reaction:
This is a ridiculous question. I am not going to answer it because it is so ridiculous.
Empathy is the right thing to do! You should feel bad for that person. We’re humans, after all.
These explanations never really helped.
*Even after reading this, I am not sure the author really gets what is behind the request.
And in mathematics, at least, priority on results is determined by the first time it appears out in the wild; for most, this is the arXiv.
Nobody is going to accept waiting a month on something that they fear may be scooped, costing them years of work and thought.
This affectively makes the arXiv obsolete for much of the purpose it has heretofore been put to.