riknos314
Joined 11 years ago
Joined 11 years ago
https://github.com/deepseek-ai/DeepInfer/blob/main/doc/infer...
So this implies they actually hit about double that during peak pricing.
The cool thing about models designed for cheaper inference is that post-training (which often requires inference as a part of the feedback loop) becomes cheaper as well, so almost certainly cheaper than models that are less optimized.
Inference is clearly profitable. The question is whether it's profitable enough to justify the training research costs.
He's still quite correct. I generally see a machine as being "programmed in" whatever is tracked in source control. So as long as your git repo is tracking changes in Java, that's still programmed in Java, even if the Java was generated with the assistance of LLMs. I may consider the thesis of this article invalidated only when the majority of repos start tracking natural language rather than programming languages as the primary artifacts defining behavior of the software.
One of the biggest metrics businesses track is repeat purchases. They don't want to give up that info because it tells them if their product is good enough to retain customers.
So the token must have some means of connecting purchases for a user across time (at least per merchant) or businesses will never adopt.
The question isn't really if the speed is bad, it's what tradeoffs you need to make to realize it. The F1 car has a very narrow operating condition and so can make tradeoffs that are completely unreasonable for vehicles that need to do other things.
From what cloudflare is claiming, Tsrs gives me more speed without tradeoffs I care about, so why not have the free speed?
It's also worth noting that more post-training does not change the model architecture. No amount of post-training on deepseek v4-flash will ever make it as efficient as deepseek v4.1-flash because 4.1-flash uses a fundamentally different architecture. Being stuck on an older base model means you don't keep up with these architectural improvements that unlock significant gains in speed / efficiency.
More performance from the same hardware means better margins. Deepseek just gave every third-party inference provider a more efficient stack for free. They also get the whole provider community working with them to make inference even more efficient.
Artificial analysis has them benching similarly on intelligence, but with deepseek 4.1-flash having a decisive victory in time-to-first-output-token and end-to-end-response-time [1].
Combined these make the interactive experience of using deepseek 4.1-flash significantly more enjoyable than haiku IMO.
[1] https://artificialanalysis.ai/models/deepseek-v4-1-flash-101...