I'm unconvinced that local deployment will be the preferred way to run generative models anytime soon.
They seem like pretty much the perfect fit for cloud - burst compute which would result in very low hardware utilisation if ran locally.
Why would it be better to have a $1,500 GPU that is weak and used infrequently, when you could share a big cluster of better GPUs shared between a big group of people, and have it more heavily utilised?
There is a philosophical argument about owning your own hardware etc, however I think the economics and performance will eventually push this to the cloud for most use-cases (most people will just get better bang-for-buck in the cloud).
They seem like pretty much the perfect fit for cloud - burst compute which would result in very low hardware utilisation if ran locally.
Why would it be better to have a $1,500 GPU that is weak and used infrequently, when you could share a big cluster of better GPUs shared between a big group of people, and have it more heavily utilised?
There is a philosophical argument about owning your own hardware etc, however I think the economics and performance will eventually push this to the cloud for most use-cases (most people will just get better bang-for-buck in the cloud).