beshrkayali
Joined 16 years ago
Joined 16 years ago
> AI risk is string theory for computer programmers. It's fun to think about, interesting, and completely inaccessible to experiment given our current technology. You can build crystal palaces of thought, working from first principles, then climb up inside them and pull the ladder up behind you. People who can reach preposterous conclusions from a long chain of abstract reasoning, and feel confident in their truth, are the wrong people to be running a culture.
I understand how people running in the same scene fall into the echo chamber effect and get gulped into the cult, but why does everybody want to be a prophet?
It is slightly weird that people accepted this new trend just like that, probably because they think this is being transparent and wanting to give attribution, but it'd be more useful like what the Linux kernel "AI Coding Assistants" page describes, something like `AGENT_NAME:MODEL_VERSION [TOOL1] [TOOL2]`, at least we get to know which model was used and/if any additional tooling on top. And `Assisted-by:` is more appropriate for that purpose than `Co-authored-by`.
Not related to the main point of the article, but I find reading long form contnet in a mono font much easier.
The models are already good enough for code generation. What we need is the harness around them actually deterministically enforcing a specific path and “leashing” the models output to be aligned with the intention of the user as much as possible. You can’t make the output of the model deterministic, but you can make everything around it to be so.
Trying to make enforcements work with prompts is like a government agency investigating/auditing itself, there’s no incentive to find problems, so you’ll always inevitably get the “All Good, Boss!”