CBLT

Yeah, I also think decision models for pruning context isn't worthwhile. You're paying to load it into context (of your decision model), in order to save on loading the same data into context (of your generation model). The only way the economics makes sense is if you need a much smarter model for the compaction, but can't pay for it to load all of that context (because it's uncached or something). That sounds like a niche but it really doesn't exist. Cheap models are good enough, and smart models are cheap enough.

Personally, I use a harness extension that auto-compacts my context just before the cache expires. Combined with the default compact-when-full, this ends up being pretty much always what I want. It's one less thing to think about.

Please don't change "cybersecurity" to "cybercrime" from one sentence to the next as a rhetorical flourish. It comes off as disingenuous and makes it harder to get your real point across.

Correct. Updated the message to brittlest. Thank you.

Sessions just broke all the time for me. The one time I dug into it, it ended up being a known issue where if codex returned over 10k characters for a turn it breaks the session. Because docker agent does not use protocols like ACP, instead trying to parse Codex's internal state in a brittle manner.

I tried not to do anything too crazy, but even use using an API key in the docker agent natively was getting me sessions that would hang and couldn't be recovered.

I tried really hard to make this work for me a couple of weeks ago but it was the brittlest harness out of any that I've used. I'd come back when it's more mature.

Sorry, private corp repos. Thanks for the offer!

We saw similar (slightly smaller effect, ~30% less instead of the 50% less you saw) improvement from trimming the number of turns. I'm not at all confident we've found the best way to do this here. We do need the thinking tokens to be spent in order to get the less obvious findings, but we couldn't get them from just increasing thinking effort. We went with forcing the model to use a tool that parallelizes the calls, and has an output floor so there's no turns with very little content read. Again, not at all confident this is the best way to do this, just what we've thrown together that works.

The actual biggest improvement is reducing the number of review rounds a PR has to go through. We were having too many rounds of drip-feeding new findings, because the AI reviewer is a subjective grader. Even if it found the same finding before, the next time it runs it might score it wildly differently. And it tends to want to score its findings across the whole grading curve, because it thinks that's more correct-looking. Our answer is to use previous rounds scores as anchors for the next round. Since we can't just use the same context and still have good performance, we needed to condense the score report into a small json we send forward.

I just reduced our AI-in-CI bill this month (while keeping or improving our KPIs), but now we have new and exciting ways to spend tokens right around the corner!

Replacement. Getting all customer requirements and migrating them to a superior service implementation.

Deleting files is easy. Decommissioning production services is not. It's too early to say for me, we've only decommissioned one vibe-coded service at my Day Job and it wasn't any worse than decommissioning any other legacy service has been. But it became legacy much faster - a couple of months.

> Some of us cared and contributed

Under a CLA.

Yeah I also thought it was strange their pareto frontier didn't include cost.

I buried the lede in my suggestion. I was saying that a future would give enough time to build or expand the fab in order to fill it. This means the company can fill an arbitrarily large number of orders - both consumers and big players - because it will just build the capacity to meet it.

I'd come at this from a different angle: we still want this to be market system, so we need to make this priced into the market. How can we price this in?

I would try to solve this by making the market structure reflect the underlying difficulty: we have to decide what capacity to produce years in advance, to construct the memory fabs. So this should be a futures market, and a capacity crunch would affect short-term-futures, but leave full term futures at the same price. Because the companies supplying the memory can just construct more capacity to fill those futures at the same cost regardless of the AI demand.