pscanf
Joined 12 years ago
Joined 12 years ago
The artifacts are only kept in your local indexeddb though, so they don't get uploaded. Or well, unless you start a collaboration session, but even then they are encrypted before uploading, and the key stays in your browser (in the url hash), so I can't see any data on my end.
But I definitely understand the concern. Would you prefer it in a Docker image? Or maybe wrapped in an Electron app?
¹ Though actually the "harness of guidelines" I've been using recently make Claude not produce _complete_ garbage.
I used the SBB app a few times, and it's actually very good, but it's still an "app experience". First, I need to set it up (download, login, add payment method), for which I need an internet connection (which I might not have as soon as I enter the country). Then, every time I need to purchase a ticket I need to open the app and figure out how to do what I need to.
Tapping in and out is superior in my view, even though I might end up overpaying a bit (though I'm probably still paying much less than what I'd pay in Switzerland :D ).
1. Even if you communicate perfectly, there's no guarantee that the LLM will "behave as instructed" and as you imagined it to. Indeed, the frustration often comes from the fact that you've said something as clear as day, yet the agent takes another path.
2. Part of the value of coding agents is exactly that you don't need to lay it all out perfectly for them. I mean, if I need to give the LLM every little implementation detail, I might as well write the code. Of course, I don't expect it to work off of "I want nice app make money", but I do expect some "intelligence" in figuring out the missing pieces.
I actually prefer that style of writing! (When it's not AI-generated ofc.) And I also try to use it in my technical blog posts. I usually re-read my drafts asking myself: "Does the reader actually care about this? Is this sentence adding something or is it just fluff?"
And actually I feel like AI text usually produces more fluff, or anyway I notice it more, but I see how it can make the result "robotic and boring".
Last month I tried my hand at finding a way to tell whether an OSS project is slop or not, based on the amount of "human attention" it received vs the amount of code it contains. The idea is that a 100k LOC project which received 3 days' worth of attention from a human is most certainly slop.
The approach doesn't work very well, though¹, mostly because it's hard to gauge the amount of attention that was given. If I see one commit with +3000 LOC, I can assume it's AI-generated, but maybe you're just the type of dev that commits infrequently.
Maybe we need some sort of "proof of human attention" for digital artifacts, that guarantees that a human spent X time working on it.
¹ I wrote about it here https://pscanf.com/s/352/
My idea was that, if I see that your project is growing 10k LOC per week and you're the only developer working on it, it's most likely vibe-coded.
I analyzed some open-source projects, but unfortunately it turns out not to be so clear cut. It's relatively easy to estimate the growth rate of a project, but figuring out how much time developers worked on it is very error prone, which results in both false positives and false negatives.
I wrote a post about it (https://pscanf.com/s/352/) if you're interested in the details.
The takeaway from my experiment is that you can really tell a lot by how / when / what people commit, but conclusions are very hard to generalize.
For example, I've also stumbled upon the "merge vs squash" issue, where squashes compress and mostly hide big chunks of history, so drawing conclusions from a squashed commit is basically just wild guessing.
(The author of course has also flagged this. But I just wanted to add my voice: yeah, careful to generalize.)
¹ Nothing is ever finished.
Ironically, that comment was added three months after I posted the article, when it was nowhere near the front page anymore, in a clearly automated and AI-driven review.
Still salty about it.
I tried to implement something like this for the search functionality, but ended up going with "old school" lexical search instead.
Mostly because, in my experimentation, vector search didn't perform significantly better, and in some case it performed worse. All the while being much more expensive on several fronts: indexing time (which also requires either an API or a ~big local model), storage, search time, and implementation complexity.
And Superego's agent actually does quite well with the lexical search tool. The model usually tries a few different queries in parallel, which approximates a bit semantic search.