I watched some explain how deepseak got good and the Chinese approach to LLM training. Really wish I could remember it. The premise was China thinks of LLMs not as a thing separate from hardware, but gains efficiencies at each layer of the stack. From Chips to software, it's all integrated and purpose built for training.
Wonder if Anthropic is making a mistake by focusing on "consumer" hardware, and not going super specialized.
Wonder if Anthropic is making a mistake by focusing on "consumer" hardware, and not going super specialized.