>The operational engineering that goes into serving a frontier model is _highly_ nontrivial, it is not commodifiable. This isn’t a SPA application.
I don't really see it frankly. Admittedly I never tried it (I only played a bit with local models) but conceptually how is it much harder than all the other cloud services that already exist? My educated guess is that building a competitor to Netflix, Youtube or Dropbox seem technically a lot harder to get running than offering inference on demand for instance.
It seems that the main barrier for competing in that market at the moment is getting your hands on the GPUs/RAM and a datacenter to put them in.
I don't really see it frankly. Admittedly I never tried it (I only played a bit with local models) but conceptually how is it much harder than all the other cloud services that already exist? My educated guess is that building a competitor to Netflix, Youtube or Dropbox seem technically a lot harder to get running than offering inference on demand for instance.
It seems that the main barrier for competing in that market at the moment is getting your hands on the GPUs/RAM and a datacenter to put them in.