Comment

Comment by sailingparrot

I don't know anything about Together quality in general, but the specific technique discussed here (speculative decoding) has no impact on the quality of generations. So you should be able to apply it to whichever model you want, and see the advertised speedup while retaining the quality of your base model.

Discussion

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton