"Altman said there are also physical limits to how many data centers the company can build and how quickly it can build them."
An H100 has 80GB of VRAM. The Highest end system I can find is 8xH100. Is a 640GB model is the biggest model you can run on a single system? Already GPT-4 is throttled and has a waiting list and they haven't even released the image processing or integrations to a wide audience. Maybe they are just unable to do inference in a cost-efficient manner and at an acceptable speed on anything bigger than GPT-4?
An H100 has 80GB of VRAM. The Highest end system I can find is 8xH100. Is a 640GB model is the biggest model you can run on a single system? Already GPT-4 is throttled and has a waiting list and they haven't even released the image processing or integrations to a wide audience. Maybe they are just unable to do inference in a cost-efficient manner and at an acceptable speed on anything bigger than GPT-4?