Ask HN: Why can't Nvidia compete in the inference market?

Hey HN! Lately I have been thinking about how the inference market will involve. We saw that llama 3 with groq was showing great results. Once question I had was why can't nvidia compete in the inference market? In the video below chamath was saying how nvida won't be able to compete in inference. https://youtu.be/1ZQ33OnGFWE?t=1254 is here just chilling his bag or are there some truth to this? What are fundamental reason that this would be the case?