Comment Comment by onlyrealcuzzo What privacy benefit do you get running this locally vs renting a baremetal GPU and running it there?Wouldn't that be much more cost-effective?Especially when you inevitably want to run a better / different model in the near future that would benefit from different hardware?You can get similar Tok/sec on a single RTX 4090 - which you can rent for <$1/hr. parent onlyrealcuzzo 1 year ago
Wouldn't that be much more cost-effective?
Especially when you inevitably want to run a better / different model in the near future that would benefit from different hardware?
You can get similar Tok/sec on a single RTX 4090 - which you can rent for <$1/hr.