theLiminator

Has anyone actually gotten good results out of this?

For the evals I've run (albeit, not super scientific), these memory systems did worse than the raw harness given ripgrep.

Yes for most devices people care to jailbreak though.

Not exactly imo, you could spend a crapton of money with white box pen testing with a bunch of frontier llms, and it might become a bunch harder to break into (it generally costs more to attack than it does to defend). Not to mention formal verification might become possible for more devices which might make it nearly impossible to find exploits.

I wish someone ran some sort of representative benchmark suite every X days to see if this occurs.

Perhaps coq/agda/idris/etc.

Imo, formal methods like more expressive/stricter type systems are key to making LLM generated code successful. Of course models will get better, but trusting the output will become much easier with a type system that proves more properties.

Yeah, local LLMs are probably an order of magnitude less efficient at a fixed level of "intelligence" if not more.

That's probably going to lead us to trusted attestation age verification.

Looks pretty cool, what steps are you taking to ensure that the backend is secure?

DataFusion is really cool, it's kind of like the LLVM of the OLAP world.

It depends on your quality bar. At a fixed level of quality, given a high reasoning sonnet vs a low reasoning opus, the low reasoning opus tends to be pareto optimal.

It's only when you need even lower levels of cost than opus at zero to low reasoning when sonnet starts to make sense at all.

This is useful research, but this particular model itself is likely absolutely useless.

What makes something `True AI`?

What's your test for it?

Please define AGI first.