Comment by martinald

FYI: nothing seems to be able to run this (easily) yet. llama.cpp, vllm etc I couldn't get working because of no support in the mainline version.