Kayou
Joined 13 years ago
Karma
121
Submissions
87
Member since
Aug 29
Joined 13 years ago
In this case the 35b a3b makes sense as it has a PP speed of around 800tok/s
https://github.com/sindresorhus/p-map?tab=readme-ov-file#usa...
https://www.spglobal.com/marketintelligence/en/news-insights...
I find Ling 3.0 tiny particularly interesting as it looks really nice for a tiny model with 7.9B total parameters, with only 1.3B parameters activated per token. Here is the result https://coolthing-ling-3-tiny.tiiny.site (sorry for the weird hosting, first I found that worked)
(it cost me almost 0 cents and done in 49 seconds)