I think the people that are saying that mean that the inference is massively profitable. From what I've seen, the margins are between 50 and 90% on an API call. It's the training that's very, very expensive!
As far as I understand, for both OpenAI and Anthropic, currently the majority of their GPUs are being used for training, with only a smaller portion being used for actual customer serving.
As far as I understand, for both OpenAI and Anthropic, currently the majority of their GPUs are being used for training, with only a smaller portion being used for actual customer serving.