kvetching
Joined 7 years ago
Karma
249
Submissions
238
Member since
Mar 28
Joined 7 years ago
AA-Omniscience Hallucination Rate (lower is better) measures how often the model answers incorrectly when it should have refused or admitted to not knowing the answer. It is defined as the proportion of incorrect answers out of all non-correct responses, i.e. incorrect / (incorrect + partial answers + not attempted).
Grok 4.2 which was just released in the API just benched the best at this benchmark.
There's a world where the labs WANT Open Source AI to cause a disaster.
Pacing the frontier just means giving bad actors an opportunity to create an AI disaster which would force an extreme response by governments. Which they seem to be begging for.
Dario's big scary example was AI that can hack the internet. Why aren't the labs calling on the government to help harden everything? Why are Anthropic's Glasswing and OpenAI's Daybreak still commercial endeavours?
To me, the only real risk of AI is hacking. Why all of this commotion about existential risk? Seems like a distraction.