andyhedges

I hate to break it to you, but the UK has had a military for some time now.

With Y2K we fixed the issue, if a bunch of work hadn't had gone into that then there would have been wide scale impact. Critical systems were fixed hence no disaster, it wasn't people fussing about nothing.

Only if you have a Beowulf cluster of tiktoks

It's interesting, the "discernment horizon" as Yegge puts it: the point where humans can't verify an answer because checking it is harder than producing it.

I'm not sure it's quite that simple.

I have a couple of problems I can't solve myself, although variants have been solved by others, so we know they're human-solvable. Verifying a candidate solution, however, is relatively easy.

For example, I've repeatedly asked frontier models (including Fable) to produce a fast squaring routine for arbitrary precision integers that beats GWNUM running under Rosetta 2 on Apple Silicon. None have come remotely close to the best human-produced implementation. After hours of iteration, Fable's best attempt was still about 4x slower than the Rosetta-translated GWNUM, despite GWNUM not even running natively.

The point is that direct understanding isn't the only way to judge correctness. We often build tests, benchmarks, and oracles that let us validate artefacts produced by people or systems that are operating beyond our own cognitive abilities. Perhaps the real limit isn't a discernment horizon, but a horizon for constructing effective evaluators. Can we layer evaluators to get even more magnification; I'm betting we can.

For the LLM it's a probabilistic set of strings that achieves the outcome, the highest probability set didn't work, try the next one until success or threshold met. A human sees the implicit difference between the obvious thing not working indicating someone doesn't want you to do it, but an LLM unless guided doesn't seen that sub-text.

So chmod +x file didn't work, now try python -c "import os; os.chmod('file',744)"

It's when they sniff the glue, then things get wild.

Isn't the what sftp is? sftp multiplexes commands and data, and is single port without all the old fashioned two port negotiation of FTP.

Yes, I'm saying it's not a good analogy

> We require age minimums for driving, voting, and drinking because we believe maturity only comes through lived experience.

Not true, we do this because the 99% of the time it's true, however there are people who would be perfectly competent and responsible to drive without living to the age of 16-18. Same with voting, there are humans who have a deep understanding and intelligence about politics at a younger age than suffrage. Equally there are people who will be reckless drivers at 40 and vote on whim at 60.

We have these rules not because sophistication only comes through lived experience, we have them because it's strongly correlated and covers of most error cases.

To take this to AI, run the model enough times with a higher enough temperature, then perhaps it can solve your challenges with a high enough quality - just a thought.