manofmanysmiles

I made something like this about a year ago. It worked until it didn't. My assessment is that an individual bypass isn't that hard, but doing it at scale is an entirely different and much harder game to play.

It's a hard game to play because they analyze behaviors across their entire network and detect behavioral patterns that are almost impossible to emulate at scale.

I'd just ask Claude about writing test specs in Gherkin. I'm going to try this Monday.

I wonder if you could train an LLM while holding the weights implementing fixed-function regions constant, so the training would produce a model that natively knows how to use these tools inside it's own reasoning steps.

I suppose it doesn't need to necessarily be in the weights, a hybrid LLM + internal fixed function blocks that aren't encoded in the weights could be quite interesting.

Combine this with latent reasoning and it gets pretty wild.

A very useful fixed function region in my mind is a symbolic execution engine, capable of only following a well-defined grammar. Perhaps a lisp would be good too?

This is close to, if not exactly what I was looking for! Thanks!

I can't see if an LLM can be put in the game loop, which is what I'd really like. Agent(s) + code in the game loop, with me steering.

Thank you!

I was transfixed as I read the first half this morning. I look forward to finishing it later today.

Of all the comments on this post, this landed the most.

I feel there is a potential to "distill" the frontier models into useful tools that can run on smaller hardware. Or distill that knowledge to produce hardware, compute or otherwise. Iff one day the frontier model become inaccessible, the tools that were carved with them may remain valuable.

They will remain most valuable if they are designed in a way that is maintainable even without LLMs. If not, they can be studied as artifacts, or reference designs and then recreated using human intelligence.

The analog I'm playing with:

You can rent time at a high end machine shop and use the best CNC machines to produce extremely valuable tools. Once you leave, you don't have access to those tools, so it is advantageous to produce durable artifacts that are valuable on their own — on a different frontier.

Imagine you're going beyond the frontier physically, exploring new parts of the earth or space. You can't bring the giant machine shop, let alone the machinery of society, the entire economy etc. What do you bring? What do you build now while you have access to the best machines to take with you?

Thanks!

My intuition is saying: the system arose naturally and intelligently over a long period of time, and even with its flaws has a lot of "load bearing legacy code" that few people understand, and new "engineers" think they'll just rewrite the whole thing from scratch and delete the "junk DNA" to use another metaphor. A little time passes and the beautiful clean rewrite collapses because of "totally unforeseeable circumstances" and the "old broken legacy system" is used to save the day. Somewhere a senior "monetary engineer" rolls over in his grave...

Any recommendations for books? Or even podcasts?

I don't have zero understanding. It sounds like you have a deeper understanding than I do.

Are there people playing AI + human vs AI + human games? I think it might be fun to play an RTS against someone where the only rules are no hacking the game itself, but writing harness, macros, any kind of stack is allowed.

Do you understand the monetary system? If so, what specific steps do you suggest I take to acquire a similar understanding?

Edit: the downvotes are suggesting I'm missing something. I think my communication style could be improved.

I am playing with similar ideas.

I am inspired by this to make a harness that intentionally does sparse checkouts, so the agents only can see so much context.

I am also playing with the idea of layering additional data structures on top of the "code" that cover different aspects. For example, a tree structure of performance requirements that has durable symbolic references to entities in the code that are especially relevant. Another tree can cover aspects of security, another legal compliance, another UI guidelines.

This makes me wonder: is it possible to publish information online that by itself appears innocuous but when used in LLM training results in "backdoor" functionality?

Thank you!

I don't know how to interpret this. My initial impression is that this has potentially imminent serious implications.

Will someone illuminate?

I am hesitant to ask, and yet morbidly curious what you are gesturing at.