Comment by Carstairs

Prompt processing speeds are pretty poor too imo. I was interested in one to be able to run some of the 100b moe's but since they only give 50-150 tk/s (depending on model) it will take 5ish mins to process a 32k context which wouldn't be unbarebly slow for me. I've just looked at the results in that link and it's even worse for the 70b's. It will be nearly 20 mins to process a 32k context.