c0rruptbytes
Joined 9 years ago
Joined 9 years ago
my company was easily spending $1m/month on tokens
sonnet 5.5 drops - we barely cracked half this month
it has to go down too, carbon is interesting, china has qwen 4 soon
coding isn’t sustainable growth - RLRVF is really effective for software development - labs are cracking the code, these models will have to get better at other things to be worth it
article seems very salty and im not sure why
also the economics is you open source something and its saves every one money and opens up more cross compile support, i bet this will scare companies to support linux more properly before linux users just start getting very good at cloning
i don't really think distilling is even that fundamental anymore (nor was the evidence ever that strong - https://msukhareva.substack.com/p/is-kimi-k3-really-distille...), the game is all RL now, and you don't need american progress for that at all (just compute and data, and the data brokers will sell to anyone no problem)
i do think there are black market proxies from china to the US trying to sell "claude" and "deepseek" access mixed together, and i do think that prompt distribution is probably going to be very nice datasets
there was never a need to rush other than for capitals self interest to control
if frontier labs wanna pace the frontier, they should be honest and return the compute back to the people
the ds4 quants were very good beating the unsloth quants https://github.com/michaelasper/benchmarks/blob/main/deepsee...
most people probably still by hand or using it to review code usually
most of the spending was actually coming from a minority of power users