Comment by jvanderbot

Shoutout to project nomad.

https://www.projectnomad.us/

its got a ton of useful options, preconfigured, integrates pretty easily, and can use any local LLM or data.