kmeh

Very interesting take.

So do you also believe that Kim Jong Un is the most intelligent man in North Korea?

Why would deep NNs or LLMs not work based on our theoretical understanding? What theoretical understanding are you referring to?

Tasks like using a CNN to detect digits has been well understood since the 2000s. The explosion in the capability of LLMs is very surprising, sure, but where is the concrete proof that such systems "should not work"?

You're assuming that your prompt is not being intercepted and rerouted by a lightweight prompt classification model.

In addition, you can make a similar comparison between Chinese models refusing to answer questions about Tiananmen Square and OpenAI and Anthropic models refusing to answer questions about the synthesis of methamphetamine; I don't think these topic by topic refusals would have real impacts on the overall performances of frontier LLMs.

Customize the exact environment of your container from the ground up (harnesses, tools, base image, packages, mounts, etc) and enter with a single command.

> HLE, AA-LCR and AA-Omniscience are now graded by GPT-5.6 Luna (medium), replacing GPT-4o, Qwen3 235B A22B 2507, and Gemini 3 Flash Preview respectively. These checks are now unified under a more capable modern model, selected for strong agreement with human judgment in our grader validation

Interesting that they chose a nano-sized model from OpenAI to be a grader for benchmarks involving knowledge and hallucination.