Comment by brettdav

Can you share a bit more about your bifrost and llama swap setup? I’m facing memory constraints and am looking for a managed model solution that will help with hot swapping loaded models and stay-warm concurrency. Ideally with prioritization.