Comment

Comment by tredre3

If you can get away with it, you really should stick to llama-server alone. When they added support the ability to load multiple models + an idle timeout to unload them, it finally did it for me.

Adding llama-swap adds a fair amount of complexity that is unpleasant to debug and its documentation isn't very good (the only real doc is the sample config file).

parent tredre3 2 months ago

Discussion

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton