Comment Comment by rynn > Please do give that a try and report back the prefill and decode speed.M4 Max here w/ 128GB RAM. Can confirm this is the bottleneck.https://pastebin.com/2wJvWDEHI weighed about a DGX Spark but thought the M4 would be competitive with equal RAM. Not so much. parent rynn 10 months ago
M4 Max here w/ 128GB RAM. Can confirm this is the bottleneck.
https://pastebin.com/2wJvWDEH
I weighed about a DGX Spark but thought the M4 would be competitive with equal RAM. Not so much.