Comment

Comment by mythz

Getting great performance running gpt-oss on 3x A4000's:

    gpt-oss:20b = ~46 tok/s
More than 2x faster than my previous leading OSS models:

    mistral-small3.2:24b = ~22 tok/s 
    gemma3:27b           = ~19.5 tok/s
Strangely getting nearly the opposite performance running on 1x 5070 Ti:

    mistral-small3.2:24b = ~39 tok/s 
    gpt-oss:20b          = ~21 tok/s
Where gpt-oss is nearly 2x slow vs mistral-small 3.2.
parent mythz 1 year ago

Discussion

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton