graphitout

For a chatbot like claude code, since the instruction prompt part (including descriptions of tools) is relatively constant for multiple users and over a long time, a lot of optimizations can be made. Even basic prompt caching gives a lot of speed and cost reduction.

We are not there yet. A bit too optimistic for today. May be in 1 year.

I added types to a python codebase (50k) using claude code and it still took some effort to make mypy happy.

Very nice. Just the thing I was looking for.

Yes. Most of the MCP based search lookup now happens over the text stage.

voicevoyage.io looks interesting. Will keep an eye.

Quite funny to see that coming from OpenAI. Nevertheless, how can they prove that?

What is the strategy to deal with pay-as-you-go services used in vibe coded apps? I am always worried about the risk of some bad loop consuming thousands of dollars in a day.

Claude gave me $50 yesterday with the release of opus4.6 and I burned my session quota + $30 yesterday itself. The code changes were very reasonable.

It is good for handling all code changes where there are no security aspects involved.

The RAG was setup on a bunch of documents, most of them were manuals containing steps about measurements, troubleshooting, and replacing components of industrial machines.

The issue was that most of these steps were long (above 512 tokens). So the typical chunk window wouldn't capture the full steps. We added a tool calling capability by which LLM can request nearby chunks of a given chunk. This worked well in practice, but burned more $$.

I do not follow. Could you clarify please? Who is going to fund and what is the criteria they are using?

optimal chunk size is strongly query-dependent - very true.

Faced similar issues. Ended up adding a agentic tool call layer on the top to retrieve the nearby chunks to handle a case where a relevant answer was only partially available in a chunk (like a 7 step instruction in which only 4 were available in a chunk). It worked ok.

Nice visualization. Would be nice to also see state variables (like stack or queue) along with the graph.

You are right.

- llm_generate() - this is the core function which calls ollama API

- get_related() - this will give a sequence of related topics for a given topic and description

- llm_get_aspect_query() - this is tricky. The interface has a fixed set of aspects for every response. Like history, related ideas, people etc. I wanted a way to create new query based on an existing query and a given aspect. This func does the rephrasing of the query.

- App.vue: handle_related() - there is a tiny bit here. When going to another "suggested" topic, it seems we have to give it some context.