Comment by qudat

I just wanted to call out that some of these quick-to-start tools are CPU only (eg ollama) which is great to play with but if you want your GPU you’ve gotta go to llama.cpp

Further, the 70B for llama.cpp is still under development as far as I know.