Hi! Latency is definitely a factor in any system, and the dashboard and paper do report elapsed time - but at the workflow level.
On a per-call basis, the wrappers are pure python ifs and such, measured in ms easily, and frankly negligible compared to the LLM call itself which will be on the order of magnitude seconds.
Where timing gets interesting is that forge will slow down workflows because the retries mean you don't error right away. Bare runs were failing fast in my experience. But on a per-call basis there's very little overhead.
I haven't detailed it simply because the order of magnitude of a single LLM call is so much higher than all the overhead put together.
On a per-call basis, the wrappers are pure python ifs and such, measured in ms easily, and frankly negligible compared to the LLM call itself which will be on the order of magnitude seconds.
Where timing gets interesting is that forge will slow down workflows because the retries mean you don't error right away. Bare runs were failing fast in my experience. But on a per-call basis there's very little overhead.
I haven't detailed it simply because the order of magnitude of a single LLM call is so much higher than all the overhead put together.