somnial
Joined 1 year ago
Karma
15
Submissions
19
Member since
Oct 30
Recent submissions
Comment · 3 months ago
Comment · 4 months ago
this is a blog post from a company that hosts open weights LLMs (https://www.doubleword.ai/). I think its possible it might have been tongue in cheek
Comment · 4 months ago
https://fergusfinn.com/blog/economics-of-speculative-decodin...
good point tho - plus for Deepseek the shared expert increases the overlap slightly
Comment · 4 months ago
true, but no reason the predictor model couldn't use linear attention (i.e. mamba, GDN etc) to predict KV caches
also it would be interesting to figure from the DSpark paper whether their numbers are consistent with the GPUs still being H800s, since they never actually say...