softwarewright

I have been using Claude CLI, Codex, Gemini, and OpenCode (Z.ai GLM-4x/5x) for quite a while. I've seen the code they all produce. I've had to add checks as forcing functions to get them to create more loosely-coupled, readable, maintainable, extensible, testable code. They often get stuck, "complaining" that the rules are too hard to satisfy, and then I show them how, make them take different approaches.

I do not mean to anthropomorphize their work, but I do need to improve what they produce. I think a lot of push-back to vibe-coded projects is that the person asking an AI Agent to make something do not or cannot improve what it produces, which goes back to the question asked. Is it worthwhile to learn programming now. I argue that it is.

So, it would be better if models were trained to produce provably correct code, and agent harnesses are improving to help do this, but the models themselves have been shown to be trained to provide the answer they think you want to hear, and declare a task done as fast as possible. They hallucinate (or lie) about running tests, even about writing code at all (sometimes).

How do cloud models "learn" to write better code? Where are the projects that fine-tune an open/local LLM to "learn" from its mistakes. I have yet to see any model improve on its own.

A lot of the best code I've seen is closed source, and unlikely to be represented in existing training data. A lot of the worse code I've see was online. Maybe newer coding models are getting trained better, but most I've tried are years out of date. Maybe my view is affected by trying so many LLMs. Very few have been great.

My experience working with programmers for decades (as an individual contributor, team lead, architect, release engineer, engineering manager) is that great programmers are rare and most programmers are good enough and a few flunk out. Since AI is trained on "the internet" it sees and learns a lot of bad habits, poor coding examples, shortcuts, oversimplified solutions, and therefore, by default, produces code that can be improved. To use AI coding agents effectively, you must be able to spot bad code, suggest better approaches, add guardrails, tests, checks, guidance, etc. to keep the agents on track and improve their output. If you do not deeply understand data structures, algorithms, idioms, best practices, footguns, design patterns, etc., you cannot help the agents provide better code.

Interesting link/content and it seems complimentary.

One thing that is missing from both of our approaches is the ability re-train (fine-tune) coding models "overnight" so that they can "learn" from the prior day and changes since their training cutoff date.

I have found some things I can do to improve my work based on this, thanks.

_Pentad idea_ -/- _MLOS relevance_ -/- _Action_

Closed autonomic loops -/- Very high -/- Adopt architecture vocabulary

Deterministic replay -/- Very high -/- Strengthen event/replay contract

Model minimalism -/- Very high. -/- Extend later to compute-placement ladder

Durable vs active population -/- High -/- Define registered vs resident capacity metrics

Standing queries -/- High -/- Future policy/watch abstraction

Provenance by construction -/- High -/- Record policy decision causality

No model/NLP in hot path -/- High -/- State explicitly as invariant

Are you saying that AI coding agents cannot produce working code? Or are you saying that it produces something that cannot be explained or understood?

If I used a macro to generate code that gets compiled and linked to code someone else provided, how is that different from asking a coding agent to generate similar code and linking to other coding agent code?

Is your concern that the output is probabilistic vs. a macro or code generator producing code deterministically? I apply engineering practices to ensure the results are suitable.

I do understand that AI is fundamentally different (and I've been studying ML in depth for more than a year). My comparison was in terms of tools increasing productivity, not how they do that.

I was a "10x programmer" on several projects and I am 10x more productive now than I was then (at least). I can create in a few days by myself (with agents) what used to take months (with a team of people).

Maybe I'm producing what some consider "AI slop" but the things I've built this year work to my satisfaction, are fun to make, and I am constantly learning new things. And I am doing this for fun but also to learn by teaching, so I build educational, visual things to share with others.

Using agents to micromanage agents is something I ended up doing.

Part of the fun is solving puzzles where the coding agent gets stuck and me then giving it technical direction. Especially when an agent says that something cannot be done in software (I show them how it actually can be done).

I am in the more motivated than ever camp.

But what if I use AI to write live demos where you can write machine code, debug it visually in your browser, punch data on virtual punch cards? What if I used AI to generate videos about this, generate the artwork, backgrounds, narration, intro music, etc. I guess I am just a meta-tinkerer :-)

I am also pursuing something similar (complementary?) to this (recently started writing a Rust-based "distributed OS" that manages ML resources in a network of heterogenous systems: varied cores, system RAM, GPU VRAM, I/O). So my focus is not so much distributed agents, but more distributed inference (and fine tuning) that would benefit agents, distributed or not. I may blog about my work soon.

Your posted link is helpful, thanks.

Thanks for the UX change; it worked well (except for links to relative images).

Is the intent just to use it to view markdown with colors, spacing?

Do you plan to provide ways to export it to something like PDF, html w/css, etc.?

Let's say I wanted to take a README from my github repo and blog about it with the pretty version in a blog post, how would I do that?

serious question: do you program in machine code or assembler or use a higher level language? do you use design patterns and functional approaches? metaprogramming, code generation, templating, app frameworks?

Each tool, abstraction, language feature elevates coding to get more done with less (more concisely). Programming is a social activity, since source code is meant to be read by humans (at least historically) even if it is just yourself later.

I'm not trying to talk you into using AI coding agents, but explain why I find them fun, and I've written a lot of code by hand over decades, from spaghetti assembler code to structureed programming to object oriented to functional, etc. Each advance in the industry has been a new and diiferent kind of fun for me.