Comment by satvikpendem

Investigate r/LocalLlama as they have various harnesses and llama.cpp forks to get models working well on your specific hardware.