blints

The entire program is a farce. It's the process generating the fraudulent "golly, we just can't find a US citizen for the job" claims, complete with fake ads designed to be undiscoverable.

"How dare people I don't like give away software"

One does not simply "buy a few b200". They come in 8 packs minimally, at >5x that budget.

I don't know that "consumer hardware" is a useful distinction anymore, it's just "what's your budget and what's your speed requirement".

It can be more than "slightly", particularly if the model you're interested in (or will be interested in in 6 months) doesn't fit on the mac studio. You also need to account for eg storing 10TB of random checkpoints, load time when experimenting, and so on. When you start actually needing throughput these are all capability gaps in practical use, not just x% benchmark differences.

If you just want to run Qwen 3.8 27B and Deepseek v4 Flash in perpetuity and that's it, there are a lot of solutions that will work and this is a fairly user friendly one.

The relevant comparison isn't one mac studio to one RTX 6000, it's a 24 channel DDR5 system, which also has ~1.2TB/s of memory bandwidth (or more when Xeon 6 compatible 8800mt/s memory becomes widely available), vastly higher prefill due to more CPU horsepower, orders of magnitude faster networking, can hook into GPU accelerators, can be upgraded etc. A baseline 384GB system from eg Puget is ~30K vs ~12K for the 256GB Mac Studio and you do get value for the money.

It's unclear to me how bandwidth scales with multiple connections. Many-to-many does not seem ideal. Daisy chaining would be fine for straight pipeline work. There doesn't seem to be an equivalent of a ethernet switch for thunderbolt 5 though.

It is. That's the mac mini. For local LLMs you would want the Mac Studio, which tops out at 1.2TB/s.

10 grand for 256GB memory. Likely double that for 512GB, but won't be available or finalized until October. Thunderbolt 5 is highest bandwidth external IO available at 120Gb/s. 1.2TB/s claimed max internal memory bandwidth.

Not exactly "future proof" for >1T parameter models but good for targeting specific lower-parameter models, or if you can rely on pipeline parallelism and run a cluster.

Their claim is even stronger than that, they have complaints about their models being used as a validation step for other model output, which is standard practice in the industry.

Most companies do not model themselves as "building on [AI model du jour]" yet. They model themselves as building products with those tools, which they consider as relatively substitutable.