Comment by smcleod

Nice work, does this work with local (100% offline) models assuming you have decent hardware and are serving them up with llama.cpp or similar?