Comment by dataexporter

Also take a look at the Python modules that Simon Willison has made available for interfacing with local LLMs. They make it much easier and effective.