There's really not much difference in the architecture ultimately.
The key thing is the data + compute resources. If you can afford to run a big H100 cluster, even temporarily, it's accessible.
The critical question though is: to what end is it profitable? If you're company X who's built one of these, it's obvious that another company could knock you off peak position next week. If that's the case, is there the incentive to continue ploughing money into training? For some big companies the answer is obviously yes because they want to dominate the ecosystem. But I'd question whether there's a large market for it in the medium-term.
The key thing is the data + compute resources. If you can afford to run a big H100 cluster, even temporarily, it's accessible.
The critical question though is: to what end is it profitable? If you're company X who's built one of these, it's obvious that another company could knock you off peak position next week. If that's the case, is there the incentive to continue ploughing money into training? For some big companies the answer is obviously yes because they want to dominate the ecosystem. But I'd question whether there's a large market for it in the medium-term.