Lots of emphasis on no kernels written and nothing recompiled but the intro and conclusion somehow suggest this is also the value of runinfra’s agents the rewrite software ?
Walk through the theory of peak decode throughout based on memory bandwidth with one weight load pass per token to provide a target for optimized decide but then doesn’t address low bandwidth utilization in that scenario at all and uses speculative decode where that theory doesn’t apply
Lots of emphasis on no kernels written and nothing recompiled but the intro and conclusion somehow suggest this is also the value of runinfra’s agents the rewrite software ?
Walk through the theory of peak decode throughout based on memory bandwidth with one weight load pass per token to provide a target for optimized decide but then doesn’t address low bandwidth utilization in that scenario at all and uses speculative decode where that theory doesn’t apply