Comment by yjtpesesu2

Oh, that's just the infra for the infra. Then use something like graphllm from matteo, and of course llama.cpp from greg, tailor you model selection to your hardware.