Comment by jumploops

My biggest problem with running local LLMs on my M4 Max/128GB RAM is the prefill latency.

I've since acquired two DGX Sparks, and it feels so much snappier.