Comment by xpnsec

Out of interest, what does your local setup look like? Is this a spare GPU you’re using or have you got something more designed for local LLMs?