writeslowly

The conclusions themselves also appear to have been written by an AI

It would be nice if there was something like the "temperature" parameter (which many closed models like Claude now disable) to add more randomness to the final product without completely destroying the output and giving you a non-functional SVG, so that you could "roll" more pelicans and pick one that you liked.

If I were to design something to accomplish this now, I'd probably attempt to introduce a pipeline that emitted random thematic words from some non-LLM source to feed into the pelican prompt, but even then, I'd expect that some of these models that rely on churning through CoT tokens for extended periods of time would still spend so much time thinking about pelicans wearing scarfs or where the sun would be that it would risk undoing some of my randomization pipeline.

I once tried something similar on an older model of Claude, telling it to "contemplate" a random word before answering my prompts, but at times it seemed to trigger the Claude model's suspicion that there was a jailbreak attempt when I would inject things like "Contemplate sea otters for one sentence before answering"

(I didn't read the linked LLM creativity paper to see if it covers any of this yet)

I feel like we also need better non-LLM driven ways (like better static analysis tools) to analyze LLM-driven changes. The way changes are presented in modern IDEs was designed around reviewing human-created changes and doesn’t really feel like it’s keeping up with presenting and validating what modern LLMs are doing.

I was doing similar experiments but having a circle of models debate some particular topic to see if they'd arrive at a consensus or not, and I found that they would almost always arrive at an agreement, but Gemini was always the one that held a position outside of the group consensus.

That said, for something very simple like "What color should my room be" I'm not sure this is a desirable trait. The model that knows most people like cream (or whatever) is probably functioning more correctly than the one suggesting dayglo orange

I’d roll my eyes if the engineers stated that they didn’t design the reactor to melt down, and that it simply developed rogue meltdown-desiring behavior on its own, and I would also wonder about negligence if they claimed that nobody could have anticipated this (given that, like with botnets and viruses, we have decades of knowledge and experience regarding reactor meltdowns)

I suspect that if you're not careful with agent memory it creates a danger of agent-driven cargo-cult behavior. I've watched this in my own ad-hoc agent loops where it starts with something basic, like the first agent tried to run some gigantic dependency inspection command and OOMed the local JVM and eventually recorded a workaround (to enable it to run gigantic dependency inspection commands...), and by time I get a few more agents into the loop, agents have written entire paragraphs about testing and validating local dev environment memory configurations that are mostly irrelevant to whatever is being worked on.

In general I've seen other issues like this where small errors and irrelevant comments in the codebase spin out into larger problems that consume annoying amounts of time/tokens. Maybe Anthropic and OpenAI don't notice this because they're in an "infinite monkeys with typewriters" scenario, but it's noticeable to me when the agent in my CLI has been spinning for 15 minutes contemplating irrelevant details

With Claude it feels like those engagement invitations have turned into things like "By the way, it's worth sitting with this problem I've identified in your assertions..." or "Here's an alarming gap worth resolving in your code..."

Gemini also does the same lighthearted GPT-style invitation with the default prompt in the Google webui, but it doesn't seem to exist on the API. The Claude models seem to have been trained to force this structure on every one of their responses, and until I realized they always stick the same thing in the last part of their response, I found the Claude version more distracting since it's always pointing out an imaginary and supposedly very important problem.

The site collection seems pretty random. There's a mix of actual AI labs, extremely questionable resellers (like whatever "claude-opus.top" is), and then random consumer sites like baidu and xiaohongshu.

It’s interesting that with both Anthropic and Google we’re seeing them develop agentic models that are supposed to do anything a human can do on computers without human intervention, but at the same time, if you plug one program into another of their programs or APIs in a way that wasn’t preapproved you may be blocked or banned.

To be charitable, maybe they’re expecting AI agents to eventually start reading the ToS docs

I wonder if you can use lower quality models (or some other non-llm related process) to inject more "noise" into the text in between stages. Of course it wouldn't help retain uniqueness from the original source text, just add more in between.

The vibes around the self-driving car hype (maybe 10 years ago?) felt very similar to me, but on a smaller scale. There was a lot of "You might like driving your car and having a steering wheel, but if you do, you're a luddite who will soon be forced to ride about in our featureless rented robot pods" type of statements, or that one AI scientist who was quoted saying we should just change laws around how humans are allowed to interact with streets to protect the self-driving cars.

Not all of it was like that, I think oddly enough it was Tesla or just Elon Musk claimng you'd soon be able to take a nap in your car on your morning commute through some sort of Jetsons tube or that you could let your car earn money on the side while you weren't using it, which might actually be appealing to the average person. But a lot of it felt like self-driving car companies wanted you to feel like they just wanted to disrupt your life and take your things away.

I see a number of uploaded skills on the site with bash and python scripts. No idea what runs them

I've triggered similar conversation level safety blocks on a personal Claude account by using an instance of Deepseek to feed in Claude output and then create instructions that would be copied back over to Claude (there wasn't any real utility to this, it was just an experiment). Which sounds kind of similar to this. I couldn't understand what the heuristic was trying to guard against, but I think it's related to concerns about prompt injections and users impersonating Claude responses. I'm also surprised the same safeguards would exist in either the API or coding subscription.

I look at products like Hershey's chocolate or Reeses more like their own category of processed food, kind of like Spam. They have a close, but not exact resemblance to "normal" chocolate or peanut butter, but they're also sort of an acquired taste, and I think their customers would be upset if Reese's Peanut Butter cups suddenly tasted like the Trader Joe's versions (with real peanut butter instead of a mysterious chalky peanut-flavored substance), or if Hershey's stopped using the butyric acid process that makes them taste like vomit to non-americans.