b1z0n

OP here. I ran the same Python codebase with different levels of type hints through Claude Code and Codex, 766 runs in total. If you would rather not go the website and poke at mesmerizing interactive results there, spoiler alert below.

I expected types to give the agent extra information, so it would be faster, use fewer tokens and be right more often. But it did nothing of that, no matter the types. Moreover, with more types it sometimes used more tokens, because there’s, well, more code.

The one thing that did change anything was giving the agent a type checker. It ran mypy on nearly every change, and no type errors got through. So give your AI a feedback tool - like type checker, to keep it in check!

Caveats: this is a case study, not a benchmark. The main codebase was written with Claude, one of the agents tested, and I had only 5–20 tasks per comparison, because tokens aren't free. At the end of the day I used 170 million of them, so you don't have to.

I replaced Figma with Claude SVG magic. I am very bad at design and have no taste, but it somehow managed to create this skill that continuously gives me images for my project thumbnails mechanically (not AI-generated images).

New to HN, is it always so empty?