Joined 2 years ago
we then apply lossless techniques only
dflash2 as spec drafter and patched ninfer inference engine (down projections for short token lengths and adding tool call related schema to drafter shortlist)
see: https://github.com/meshapplied/goethe/pull/4
benchmark numbers are measured using see bench verified traces at avg input 28k and concurrency 1
we then apply lossless techniques only
dflash2 as spec drafter and patched ninfer inference engine (down projections for short token lengths and adding tool call related schema to drafter shortlist)
see: https://github.com/meshapplied/goethe/pull/4
benchmark numbers are measured using see bench verified traces at avg input 28k and concurrency 1