Comment by Brandon_Chen

We have been developing an on-device model recently. For macOS, we are using Ollama to host the on-device model on Mac. We have not yet found a perfect solution for the Windows version. Thank you for sharing Llamafile; we might test it for the Windows on-device model.