Comment by vardalab

I would probably run qwen 27B or qwen fn but I would run the actual inference on one of my other boxes and not on actual desktop so initially it could be sufficient if you just enabled openai or anthropic compatible endpoint.