Comment by busfahrer

I am eyeing one of these specifically for this use case, could you please post roughly what kind of tokens per second numbers you get for text generation for this 27B model?

edit: and which quant you are using, please :-)