I've never run one of these models locally, but their README has some pretty easy to follow instructions, so I tried it out...
> RuntimeError: Found no NVIDIA driver on your system.
It's true that I don't have an NVIDIA GPU in this system. But I have 64GB of memory and 32 cpu cores. Are these useless for running these types of large language models? I don't need blazing fast speed, I just need a few tokens a second to test-drive the model.
> RuntimeError: Found no NVIDIA driver on your system.
It's true that I don't have an NVIDIA GPU in this system. But I have 64GB of memory and 32 cpu cores. Are these useless for running these types of large language models? I don't need blazing fast speed, I just need a few tokens a second to test-drive the model.