rajveerb

An argument against having a separate hypervisor agent.

A separate hypervisor agent at least doubles the cost because (1) the underlying model needs to be the same so that we get the same degree of intelligence, and (2) it needs to have the same context + more tokens for the work it does.

i read your blog, what are the implications roughly of different approaches of treating tool call acceptance in the context window?

is there a github repo for this project?

awesome project, thanks for sharing. can you add a page on how to consume it, it's a lot to take in currently. i would like to come back to it!

it would be cool to have this task (or some variant) in a benchmark.

what sort of compiler?

As someone who has spent quite a lot of time on inference, I would a add a small note:

Deployment looks very different for MoE than dense style models so I would say that it is more nuanced than "inference memory reqs remain the same". Memory can be very different for MoE style models.

haha sounds like it.

glad that you found it useful that was the intent.