we achieved this on stock nvfp4 model which is within 5% of fp16 version
we then apply lossless techniques only
dflash2 as spec drafter and patched ninfer inference engine (down projections for short token lengths and adding tool call related schema
to drafter shortlist)
we then apply lossless techniques only
dflash2 as spec drafter and patched ninfer inference engine (down projections for short token lengths and adding tool call related schema to drafter shortlist)
see: https://github.com/meshapplied/goethe/pull/4
benchmark numbers are measured using see bench verified traces at avg input 28k and concurrency 1