~100t/s prefill, ~15t/s, dropping to ~10t/s later with 64k context.
The issue is I have yet to find a useful agentic local llm that I can run on this machine.
Just given a relatively simple task on a swift app, took 25 minutes, brainstorming like crazy but can not decide on what to do. Eventually I killed it. GPT 5.6 sol-medium took 3 minutes to complete the same task for reference.
~100t/s prefill, ~15t/s, dropping to ~10t/s later with 64k context.
The issue is I have yet to find a useful agentic local llm that I can run on this machine.
Just given a relatively simple task on a swift app, took 25 minutes, brainstorming like crazy but can not decide on what to do. Eventually I killed it. GPT 5.6 sol-medium took 3 minutes to complete the same task for reference.