There are quite a few local LLMs released in the past few months, do you run them? If so, what are your usecases?
Personally, I find them good, but very slow (RTX 3050, Mistral 7B) and hard to make them output in a consistent format (JSON, bullet points). GPT 3.5 makes it look like a pointless exercise from a speed and consistency perspective.
Any usecases for Local LLMs apart from them being local so we can feed sensitive documents?
Personally, I find them good, but very slow (RTX 3050, Mistral 7B) and hard to make them output in a consistent format (JSON, bullet points). GPT 3.5 makes it look like a pointless exercise from a speed and consistency perspective.
Any usecases for Local LLMs apart from them being local so we can feed sensitive documents?