kfsone

Sure, but what it is that makes you believe the compiler shouldn't be the one doing that analysis? Should it catch

``` ((void()(void) free)(buf); ((void()(void) free)(buf); ```

It's been my experience people invested in static analysis think of it like some wholly additive extension.

Compiler-independent static analysis is code applying rules. Someone has to write all the rules, align them with the language and the compiler. The user rarely reads the static analyzer's code or full rule definitions and is blissfully unaware of the full set of disconnects, inconsistences, and gaps between the coverage of the two.

They only notice divergence in the analysis when it generates a false-positive warning or error.

Static analysis adds compilation overhead. Sure, but I'd hazard that re-reading and in to some degree repeating the parsing/translation process that the compiler is going to do also introduces overhead.

There are some languages that demonstrate sta-by-compiler capabilities with heinous compile times, but it doesn't have to be that way. We can engineer better, but at some point it requires a price. Better comments, boilerplate of some kind or another, annotating intent over idiom.

A complex type system isn't ideal; with the right set of primitives you can achieve sophistication without complexity.

``` Freeable buf = malloc(SIZE); free(buf); // compiler error, you didn't check buf isn't valid. free(buf); // compiler error still if you fixed the above, free makes buf invalid. ```

"Making the compiler do STA makes it slow". I don't think that's proven one way or the other. There are examples both ways. "complex" type systems frequently have slow compilers, but if you look more closely that's usually because they're trying to compensate for the disconnect between organically emerged complexity in their under-designed type systems.

Also exploring mechanisms that try to pre-emptively keep attention-draining distractions/anti-patterns out of the context, things like when a model edits a file, we take the cache hit of removing the stale versions it read to make the modifications, replacing them with a reference syntax that the model can access in a sort of sandboxed auto-fork of the context.

That's going a little slowly because I'm trying to strike a balance between working 'reasonably' with extant models, and providing a mechanism to SFT/lorafy a model to make best use of it.

To a software engineer, a (current) LLM is a stateless algorithm that performs an idempotent transformation on a large numeric input.

People who think it's a system that thinks and reasons have confused the agentic harness, perhaps forgotten(?) layer0[0] is a seed, the inference engine sets to a concrete value when the caller leaves it as 0.

They probably work on (current) AI software by repeatedly writing prompts like "DON'T READ THE FILES IN /tmp. SOME OF THE FILES IN /tmp ARE VERY LARGE. DUE TO THEIR SIZE, YOU ARE NOT TO READ THE FILES IN /tmp." and wondering why the model becomes obsessed with files in /tmp 100k tokens into every conversation.

LLMs have great potential. So, it turned out, did uranium, just not as chewing gum or a hair pomade.

There are good ways to leverage LLMs, but there's a lot more load bearing wait on that word 'leverage'. Something needs to do the leveraging, and do it well.

I'm experimenting with my own harness at the moment, currently codenamed Murder because I call the individual contexts/agents 'crow's.

The fundamental unit of it is what I call 'intrusive harnessing', where the harness actively manipulates the token stream so that significant quantities of tokens are only ever exposed to Layer0 when it's useful for them to be present.

For example: the full instructions for shell-tool calling aren't in the system prompt diluting attention while the model is reasoning/discussing what kinds of cat picture you want to put in your app.

My approach is more like dev-branching, and it seems to be working way more effectively than compaction or simple aggressive sub-agenting.

As soon as the harness sees the model is inferring a shell tool call, I stop the inference, mutate the context so that the full set of instructions/examples/guidance for shell tool use are inserted. Once the model has inferred the tool call, I curate the output it gets back. I ask the model to evaluate the output - good or bad - and give it a chance to accept/retry, before allowing the tool-call and output into the original context.

Does it use more tokens? Yes, although we're only mutating at head, so in a long-horizon context, it leans heavily into cache, just not the way anthropic/openai want you to realize you can.

It sounds like compaction but it doesn't come with the nasty brainwash experience where you just need the agent to fix that one last thing, it compacts and the agent comes back a paranoid delusional mad max.

``` <|system|>You're an AI agent. You do agent things. <|system|> ... there's a list-dir tool and a shell-call tool ... <|system|> ... memories ... <|user|>It doesn't look like it ran. <|reason|>I should look and see if there are any errors in the log file.<|agent|>I'm going to read the log file to see if there are any errors. <|tool-call tool=shell-tool ```

We stop there, and splice in the detailed instructions for the tool the model was about to predict. I'll use <|ALLCAPS|> to denote harness-generated pseudo turns.

``` ... as before ... <|agent|>I'm going to read the log file to see if there are any errors. <|SYSTEM|>Shell Tool: ... shell-type=bash, zsh, fish, pwsh on this system. Preferred shell is ... Additional arguments ... Pagination ... <|tool-call tool=shell-tool ```

the model finishes out the call. On windows, with a typical harness, this frequently goes like this:

``` <|tool-call tool=shell-tool|>Get-EventLog ... | head<|tool-call|> '''tool-result error: unknown command: head ''' <|agent|>Ah, windows doesn't have head. Let me just read the whole log. <|tool-call ...|> '''tool-result ... 500k tokens ... <|agent|>I see some windows log events but you didn't ask me a question. Daisy, daisy? ```

With Murder it goes like this:

Rev 1 ``` ... prefix as before ... <|tool-call tool=shell-tool ```

Rev 2 ``` ... prefix as before ... <|SYSTEM|> ... how to use shell tool; shell-related memories and rules ... <|tool-call tool=shell-tool shell=pwsh fence-vs-escape=true|> '''pwsh Get-EventLog ... | head ''' '''tool-result error: unknown command: head <RESULT>Your tool call terminated with an error, ... ... structured response required ... options <ACCEPT /> or <ACCEPT> <WITH> annotation </WITH> </ACCEPT>, <REDO> ... </REDO> <RETHINK> ... <|reason|> windows doesn't have the head command. Let me try reading the whole log. <REDO><TOOL-CALL> ... replacement tool call ... </TOOL-CALL> <WITH> ... model note ... </WIDTH></REDO> ```

I take that feedback and loop it, so, Rev 3: ``` <|system|> ... how to use shell tool; shell-related memories and rules ... <|agent|> ... prefix as before ... <|SYSTEM|> ... as before ... <|agent|>{prev_cmd} failed, because windows does not have a head command. Let me try reading the whole log. <|tool-call ... no head ...|> '''tool-result ... first few lines of result ... ''' <|system|>Your tool call succeeded but generated 446,219 lines of output. Only the first 5 were listed. ... structured pagination / retry / rephrase options ...

```

It then repeats while the model figures out the right command, figures out which filters to use, but the harness effectively immediately guides the model to do an immediate [optionally self-adversarial] review of the command against the output until the model concludes that the result is useful by various criteria. That doesn't mean successful - sometimes what is superficially an error (no such file or directory) is the answer you were looking for.

Let's say it takes the model 3 more turns to figure out how to use event viewer, and finally it <ACCEPT>s.

Here's the win, the outer main context - the one we're going to keep growing as you work with the agent, looks like this:

``` <|system|>You're an AI agent. You do agent things. <|system|> ... there's a list-dir tool and a shell-call tool ... <|system|> ... memories ... <|user|>It doesn't look like it ran. <|reason|>I should look and see if there are any errors in the log file.<|agent|>I'm going to read the log file to see if there are any errors. <|tool-call tool=shell-tool shell=pwsh|>Get-EventLog ... | ... | ... '''tool-result (use ref-tool id=A401U8X593 for full transcript) Event ID | Last Occurred 1010111 | 3 weeks ago ''' ```

We used a lot more tokens. How can that possibly be good?

It's happening at the end of the context, so the cache comes into play very effectively.

But if we'd let all that derp into the context, it would be a potential attention sink degrading the value/worth of every subsequent token.

The pattern of try-thing-fail-try-solution-fail-try-win appears to be an incredibly strong pattern for most agents.

Fundamentally: When you're 3 prompts down the line and there's the imprint of the model doing "somewindows command | head" in the context with the model litigating it and fixing it -- that meta-pattern will drive the model to predict more of these patterns. It's going to repeatedly eff-up the exact way it saw in its training material.

When I try to get Claude/Copilot to work on this codebase, they freak out. The hyperbole/marketing pitch the agents were trained on and is built into their inner prompts cannot seem abide the idea of stopping an LLM mid inference. They seem driven to perceive an LLM endpoint like a 911 call you can't just go quiet on.

I have a mechanism for non-parallel sub-agents ('maggots', their job is to curate a large body of work whose full text is irrelevant to the main context). Basically just a tool call, but every time Claude or GPT have been near it, they've broken it, forcing it back parallel so they can send the invoking model a notification that it's child has been spawned and the parent should call the 'check-result' or 'wait-result' tool when they're ready to receive the results.

One of my test architectures is running against a solo Unsloth Studio instance that can only load one model at a time. It really doesn't react well to having you load the coding model to start your sub-agent work and unload before the model has generated its first token... :)

There's a reason engineers are prissy about people coming along and saying "I write code, I'm an engineer" that people periodically try to sand-paper away.

Engineers don't just tie a sheet to a rock and throw it off a cliff and call themselves aerospace engineers.

They do full diligence on the theory, math, physics, material science, fluid dynamics, etc, and plan a controlled series of tests specifically designed to verify/challenge/disprove their concept and the theories behind it.

Sure, there's a team member ultimately responsible throwing half a dozen rocks off a cliff in the first test.

A technician.

The guy who throws the rock off the cliff is a technician.

I was the kind of shell guru whose teeth itched when they saw someone doing `grep | awk`. Then I had to try and bring Windows into a Mac+Linux CI system under Jenkins groovy files that were full of `"""sh` fragments, shell scriptlets wrapping python, etc, etc, and I don't bat (I don't groovy either).

Option 1: Learn to bat and try to translate. Hmm. No. Just no Option 2: ?

Pwsh core had just come out. My immediate thought was that it would be great material for an anti-MS-ragging blog post, but then a line leapt out at me from one article I was glossing over: "... POSIX Terminal Shell Spec ...".

I still wanted that anti-MS-ragging blog material, so I decided to try and use Pwsh as a Rosetta stone until I got to a point I could convert to a real language.

But things just began to click for me. It was like going from Perl to Python - suddenly everything is an object and you can interact with everything* that way.

There's no need for grep or awk or sed in pwsh, because the output of a shell command is an object -- a string (or []byte). It has methods.

(netstat -an).replace("192.168.86.", "10.0.100.")

6 years later, pwsh is the default shell on my Mac, Ubuntu boxes, lab vms, ... everything but my docker containers unless I'm feeling feisty.

Careful - you start from the POSIX terminal spec, you think maybe it would be interesting having objects instead of raw text streams, and next thing you've reinvented powershell...

(*35 years living and loving sh/ksh/bash/dash etc, only tried pwsh so I could write some comparisons and slag off MS a bit; now it's my default shell on everything)

It doesn't sound like you were trying to launch a product, but doing an experiment and someone threw you under the HN-spotlight-bus :) Is this a "see what I can achieve with LLM coding" or is this "build this and see how much of the coding can be accepted from LLMs"?

On the summer solstice, with the bulb going on shortly after sunrise and burning out just before it started to get dark? :)

Using mosh to do the same (which is helpful if you're on a laptop and you want to survive lid closes or ip roaming)

I thought that was where I was going to end up, but that's where Claude's remote control or copilot's agents tab spared me.

The hitching is compounded for me because I tend to run the agents in squads: the agent I talk to operates a strict no-coding 'producer' mode, it tasks a sub-agent to do the research or coding, then the results go via a file to a critic or review agent; keeps the producer context very minimal and lean. Not convinced it's as necessary as just starting new contexts frequently with Fable etc.

My general rule is that I won't commit code a human hasn't seen/reviewed to production codebases, and I know I won't maintain that rule if I have to read all the slop that gets generated first time round without an AI reviewer pass.

So far my producer skill has survived 4.6 thru fable in succeeding to treat the review/critic output skeptically, as a likely yes-man or team player.

The key is to remember that, as of Fable, the size of the training corpus segment representing people responding to AI-generated content is still relatively tiny. Telling Sonnet 4.6 "this is code an agent produced" has a near decorative effect with no apparent significance, Sonnet 4.8 shows some misgivings, and when I experimented with Fable it seemed to do well at anticipating the kind of slop 4.6 would throw you.

Interestingly, to me, telling Fable that code a previous Fable agent wrote was AI generated seemed to raise some kind of "I'm being benchmarked" flag; expanding the reasoning finds it being evasive and mistrusting; look past the null derefence because this must be a trick question type thing.

So it's like mosh + tmux?

> On an early checkpoint of LFM2.5-2.6B, 10.2% of completions on hard math and coding prompts produced repetitive loops. After Antidoom training, that rate fell to 1.4%, with eval scores improving across the board as a direct result of reduced looping.

That's a 9% reduction, and it doesn't really discuss applicability over longer contexts, different language/language familes/programming, or context/rope length aspects.

Several families - esp Llama and gemma - seem to be easily doom-inclined during expansions (I realize this is a pathological misuse of the tooling, but it's what people do): prompt: "add comments to this file", +1000t file read, [+3k reasoning], +800t edits, +1100t file re-read promoted by tool, prompt:"pep8 format it", [+2kt reasoning], +700t edits, +950t file re-read.

The unmodified/repeating sequences in those file reads, the differences between them, is a honeypot for attention.

I'd non-scientifically concluded for myself that this was tied to an anti-pattern for rope. In the first portion of the context inference needs attention to fall in certain places to produce the correct suggestions for the first round of edits, and somehow largely refocus attention to the subsequent version of the file in the context to correctly apply the subsequent edits, not getting distracted by the tantalizing pattern of repeated sequences and disrupted sequences

Using Gemma4 e2b I can repro this in LM-Studio, and fix it by hand-editing the context so that there's only the latest version of the file being worked on up at the start of the context, or replacing the original reads with something like:

" [tool name='read-file' path=...']... elided, see file.v1.py if you genuinely need to revisit the original state...[/tool] "

The handful of times I've tried this over the last 3 years it hasn't cared to discover file.v1.py didnt exist, limited samples = limited confidence.

I suspect the next real advance will be the LLM equivalent of thin-client/fat-server evolution, a sort of "local lora" system that forms the unique part of your own agent(s) distinct from the underlying static model, and capable of dynamic learning/processing.

Could it be cloud-based? That changes the risk and scale calculus in a way that's going to take a long time to get funding into.

Maybe, but I think the downplay is worth it to emphasize that the things have to be done. The point is precisely that token generation is a phenomenal power source but it does not give you anything but debt unless you build the tools to leverage it.

None of the big players involved right now seem to be doing anything to disarm the perception that the LLM does all the work.

I feel like we're at a state fair 180 years ago having just seen an impressive demonstration of small animals momentarily coming back to life, someone in the crowd said 'eternal life' and everyone lost their minds trying to give money to the 'inventor' on the stage who applied the electric current.

Ask some lay people or junior devs how they feel about LLM mistakes, listen for answers about "learning" etc. Do they realize the model doesn't change? The same set of weights and tensors are deployed to every GPU hosting it. If you use a power tool that lets you specify the temperature and seed, it will generate the exact same output time after time.

"But I told it to always speak like a pirate, and it does": No, you signed up to pay for the model to be told to speak like a pirate every time you use it.

There - that's the curtain I'm talking about that you're lifting/tearing. I think momentarily downplaying the potential of LLMs themselves is worth it to expose that.