Fishkins
Joined 15 years ago
Joined 15 years ago
To be more precise, this article says auto mode blocked 89% of dangerous commands in their testing.
A previous article[0] said it blocked 83%, but presumably it improved since then.
0: https://www.anthropic.com/engineering/how-we-contain-claude
It lets you put the non-car stuff closer together, so you're traveling less distance to get to the same place. It requires urban design, not just a single person switching between modes of transit.
(Although switching to cycling can often make transit both faster for you and the people around you in a city because you aren't as affected by traffic and don't create as much traffic)
There many decent options (cloud VMs, local VMs, Docker, the built-in sandboxing). My point is just that folks should research and set up at least one of them before running an agent.
I never said "permissions", I said "sandboxing". You can configure that in settings.json.
https://code.claude.com/docs/en/sandboxing#configure-sandbox...
Have you noticed any change in that trend in the past year or two, or is it continuing to get better?
This is important no matter how experienced you are, but arguable the most important when you don't know what you're doing.
0: or if you don't want to learn about that, you can use Claude Code Web
This is broadly true, but not comparable when you get into any detail. The mistakes current frontier models make are more frequent, more confident, less predictable, and much less consistent than mistakes from any human I'd work with.
IME, all of the QA measures you mention are more difficult and less reliable than understanding things properly and writing correct code from the beginning. For critical production systems, mediocre code has significant negative value to me compared to a fresh start.
There are plenty of net-positive uses for AI. Throwaway prototyping, certain boilerplate migration tasks, or anything that you can easily add automated deterministic checks for that fully covers all of the behavior you care about. Most production systems are complicated enough that those QA techniques are insufficient to determine the code has the properties you need.
https://www.aiweirdness.com/dont-use-ai-detectors-for-anythi...