vipshek

I agree with one of the sibling comments that determinism isn't necessary for certifying a product. All engineered products operate under uncertain conditions; we define standards for how those products ought to respond under those conditions and verify them under measurement. Consider robot vacuums, for example.

I also agree that qualitatively, this technology seems different than the others. However, I feel that people tend to overly fixate on their internal stochasticity. Even if LLMs' internal mechanism is nondeterministic, shouldn't we be able to verify their "side effects" aren't harmful? Of course, "harm" is subjective and at this scale, the most effective way to verify behavior is probably some kind of LLM-as-judge...

Anyway, in this case the problems have occurred while actually running the evals themselves, so again, we're in a situation where we can't even confidently test these things and know that they won't cause harm in the outside world.

In the physical world, it seems like when an tool/device/instrument causes harm (or is used to cause harm), we assign blame to either the user of the tool or its creator.

When do we blame the user? When the tool is operating as intended by its creator, and we agree the tool meets certain quality standards and isn't defective.

When do we blame the creator? When the device doesn't meet those quality standards and reasonable use caused harm inadvertently. For example, for consumer devices, certifications like UL/CE are used to define acceptable performance levels and safety standards.

Maybe we need "quality certifications" for AI agents - essentially eval suites that demonstrate those agents won't cause harm under reasonable patterns of usage. Right now, these eval suites are run best-effort by the labs themselves.

The tricky thing is, a lot (all?) of these recent safety incidents have occurred while evaluating these models! This suggests we need much more rigorous standards for how exactly an eval can be run. Perhaps all of them should occur in truly air-gapped environments... though that may run counter to evaluating agents in a realistic way.

Regardless, it feels like the "industry standards" common in, say, electrical engineering and other disciplines are sorely lacking here. Unsurprising given how new these technologies are, but concerning since the blast radius for this technology is likely much larger than other technologies we've encountered in the past, except maybe nuclear technology.

This article is ostensibly about biology but it's really about pedagogy—how traditional education squeezes out the sense of discovery and turns many subjects into rote memorization exercises.

This reminds me of the pedagogical philosophy of Seymour Papert, who was heavily influenced by the ideas of Jean Piaget. Piaget's "genetic epistemology" argues that knowledge and understanding is created by interacting with environments, which traditional educational approaches fail to provide.

Papert combined Piaget's ideas with the emergence of computing to argue that children should be taught subjects in a hands-on, exploratory way - and not just for teaching computing! The idea is that by programming in simplified languages, children can discover ideas in subjects like mathematics and grammar - deriving them as they try to solve problems instead of having them dictated to them.

Suppose that biology was taught in a game-like environment where students were designing cells or organisms in some way. Maybe the game could be structured so that each organelle could be "discovered" by the student as they designed the cell to survive in some environment. Perhaps that'd make the purpose of each cell component more grounded and memorable.

Papert's book Mindstorms covers all this in detail, and I highly recommend reading it. I feel like it's especially important today given fears of how AI will affect childhood education. At best, I hope that computing can be a boon to education instead of a detriment if it's woven into pedagogy thoughtfully.

This is intriguing. The two relevant features seem to be 1) realtime collaborative multiplayer conversations and 2) conversation-as-document - basically, letting you comment inline in an agent conversation.

For (1), the main value I'd see is in mentoring junior engineers or less technical contributors on a team. If someone puts up a PR with sloppy results, you could actually jump into the thread that produced that PR and see how the results came about, or even coach that contributor on how to do better next time. Also might make it easier to hand off work from one person to another - right now most coding agent sessions are user-local.

On (2), I frequently find myself consuming agents' gigantic text responses and tediously writing 8-bullet-point responses to guide them. It's pretty exhausting. I could see inline comments providing much better ergonomics.

All that being said, Zed has largely fallen out of the conversation for "agentic coding tools", and so this feels like their attempt at creating something like the Cursor Agents Window, Codex, or Claude Code. These two features seem compelling, and I understand they're even compatible with other coding harnesses. But I don't know if there's enough there to have a defensible product; if these features are excellent, others will clone them eventually.

Regardless, would love to give this a shot!

> This website has been temporarily rate limited

Feels a bit ironic... though this website is hosted on Cloudflare Workers so using an American company anyway?

I don't have much to say about this post other than to vigorously agree!

As an engineer who's full-stack and has frequently ended up doing product management, I think the main value I provide organizations is the ability to think holistically, from a product's core abstractions (the literal database schema), to how those are surfaced and interacted with by users, to how those are talked about by sales or marketing.

Clear and consistent thinking across these dimensions is what makes some products "mysteriously" outperform others in the long run.

I find this perspective bizarre. Though I'm not happy about it all being centralized, the closest thing we have these days to the very niche phpBB forums of the 2000s is various subreddits focused on very specific topics. Scrolling through the front page is slop, sure, but whenever I'm looking for perspectives on a niche topic, searching for "<topic> reddit" is the first thing I do. And I know many people without any connection to the software industry who feel the same way.