atomon
Joined 17 years ago
Joined 17 years ago
Yes, very long workflows with no checks in between will have high error rates. This is true of human workflows too (which also have <100% accuracy at each step). Workflows rarely have this many steps in practice and you can add review points to combat the problem (as evidenced by the author building 12 of these things and not running into this problem)
There have been a number of solutions popping up to address this problem, and I think the need is very real. Decomposing these LLM tasks into subtasks seems to be one of the best ways to work around the shortcomings of LLMs in production apps (hallucinations, context window limits, etc). But then you end up with complicated pipelines that are difficult to debug, improve, reason about, etc.