LeonidBugaev

Having the CLI allows you to have the huge application with the big surface be available to the AI agent and so they'll be able to learn it on demand. For example my app has more than 1,000 help pages. There are no other ways to load all this information into the CLI context and to be frank it will be quite stupid. Instead I use progressive discovery.

It first reads the original help message to understand which stage of the flow it is right now. It auto-discovers the topics through the error messages and through the various hints. It has an Elasticsearch-like search inside its own help command so it's a full self-contained application with self-discoverability, which has a graph, ontology and the full developing flows inside of it. All of those parts are automatically given to the AI only when it actually needs them.

And to be frank if you look at the latest YCombinator batch, all of the companies are building the custom harness. What I'm calling above CLI is actually a custom harness, which is running inside someone else's agent group.

Having remote MCP allows me to give limited read-only surface to clients which not supposed to run any CLI commands. And while MCP has similar patterns, like bundling docs and prompts, it is not as flexible as CLI and does not allow huge scale.

Difference is subtle, but crucial. And it is so easy to be confused, while measuring small model performance, like various Flash variants or local models.

Smaller models became quite good at doing the work under the well specified goal, like fixing the well specified bug. But can it find the bug, without you giving it any clues?

In my benchmark the answer is mostly No. Being able to invent the questions - thats the real intelligence - and thats where the big models show true strength.