Comment by platinumrad

You can just host the inference locally... Baseten and Fireworks already do that for you.