blixt
Joined 15 years ago
Joined 15 years ago
I built something for this and also found that the idea of a "guardian" core that can detect misbehavior (crashing etc) and roll back a messed up state really helped. But it's a lot of work to get a harness to behave well, and each model reacts differently to the harness limitations, so I found more success in building a meta-harness instead (just run existing harnesses inside a common UI and inject all capabilities).
I don't think it's productive to tell students today they must spend all their time understanding the code. Because it's clear now so many people will bypass that, even myself included. Some of the most fun I have with software these days is figuring out how to make it without looking at the code (for now on side projects only, see below).
I agree we're not quite all the way there yet to do this with large production systems. Yes, LLMs are highly fuzzy and non-deterministic things, but so are electrons! As computers improved we handled random bit flips that would occur due to a large number of unpredictable reasons. In either case, we have to reach a high enough confidence and redundancy bar that it doesn't hinder our productivity.
Raising confidence and redundancy in code output from LLM is still a nascent field. We're learning some things like "maybe unit tests don't work as well for LLMs as they did for humans", and "if the AI can self-evaluate in a loop against a factual number, it does a lot better".
But yeah, in this rapid wave of change I would consider "write the code yourself" more and more similar to "know how your CPU does branch prediction so your loops perform better", and the majority of our efforts will need to go into raising our confidence in this new fuzzy shape of software.
[1] Part 1: https://www.youtube.com/watch?v=w5QSVhgrqVE
We have many agents per employee working in parallel pushing way more commits than was humanly possible before AI, triggering GitHub actions a lot more than the workflows were built for, causing Actions costs to escalate (they really aren't cheap if you compare to hosting it yourself), meanwhile working with YAML workflows is just a pain, and just writing code would be so much more fun and AI compatible[1].
At the same time, GitHub has about ~3 different PR review UIs? And they're all half-bad? Any decently sized PR triggers their "optimized for large PRs" UI which jumps around randomly in my experience. If you don't get that UI and keep the scrolling one (there's an old and a new one btw) then god forbid you click a line number because at some point your browser will randomly scroll back to that line and it won't unstick. Now Linear[2] (and others) is replacing the PR review experience for the agentic era.
I'd love to see a solid AI first Git + CI + reviews.
[1] Cloudflare CI https://blog.cloudflare.com/ci-workflows/
[2] Linear PR reviews https://linear.app/changelog/2025-01-23-pull-request-reviews
[1]: https://www.ikea.com/us/en/p/trotten-cabinet-with-doors-whit...
(I had originally replied on safety of letting LLMs skip questions but I don't think that was your point so I removed it.)
Furthermore, believing that the only thing saving you from disaster is Claude deciding to ask you a question is not a great conclusion either. You need guardrails in the power you bestow upon Claude from outside, not from inside.
Meanwhile, this article was written by Claude and has sentences like "Which cuts less far than it looks.", which I doubt Claude stopped to ask about.
for (...) {
el.style.height = `${something}px`;
whatever.value = el.style.offsetHeight;
}
This forces the browser to recalculate layout multiple times in a single frame. Separating layout changing code from measurement code will help a lot here (most frameworks out there have solved this so we don't have to be too concerned about it though).What is least surprising? That INTEGER implicity accepts 'hello world' without error, or that you can't insert such a value unless you use a keyword like NONSTRICT or a type like ANY?
I would wager the vast majority of SQLite users if asked would probably not expect it to work.
All put together, paying ~$60 to get a hit-or-miss report seems a bit excessive, but obviously as the models they use under the hood get better it becomes more and more worth it, assuming they also improve their grounding/search capabilities.
I'm a big fan of Sakana though, and have followed David Ha / @hardmaru since the world models papers (with the racing car game and the Doom clone), which were incredible at the time.
Some nice topics to talk about instead:
- When to use an encrypted value (and symmetric or asymmetric), vs. a random (but secret) value, vs. a signed value (readable but not tamperable)
- Where to put these values (memory, localStorage, cookies)
- How to make sure these values don't last forever, and whether you need to be able to revoke them (make them invalid before their natural expiration timestamp)
I've been using the c, cw, and cr shortcuts for a long time but today I wished to switch to another account and wanted to keep the simplicity so I built this.
It's been mostly coded by Fable 5, with strong guidance by me. I've vetted the code and made sure the code it executes is readable in one file and that domain is secured, script is checksummed, project is open source, etc. but as always use your judgment and double check what you download from the internet!