Comment by AbsurdCensor

I think under Lemonade you can do that, especially with AMD systems, using the models that allow for hybrid operation. Prefill happens on the NPU and token generation happens on the GPU. Not sure if this is faster or better than just doing it all on the GPU side.