Xeoncross

> "Man you Apple Fans really live under a rock... ...refurbished 3090 is 2500"

Before you make assumptions, there are valid reasons for the choice I made:

I paid $1k for a refurbished M1 Max 64GB macbook that can run Qwen 3.8 27B at 24tok/sec (looks like they are about $1.5k on ebay). In my case, Apple was a lot cheaper (used) if you're looking at price-per-token. Yes, it's almost twice as slow as a 3090, but it's also half the price /performance ratio.

Qwen 3.8 27B Q4 with 64k:

3090: ~40tok/sec

M1 Max : ~24tok/sec (MLX)

In fact, the Max can even do 265k context which isn't possible with a single 3090 last I checked (without dropping to Q3-Q2)

We're getting close to local AI being really useful. Before the M1 is dead we might have already seen a huge shift. In my mind, there are two types of macs: those that have the bandwidth to run local models (Max & Ultra) and those that don't (Base & Pro chips).

The generation doesn't mater as much to me as the memory bandwidth. When that day comes, I'd rather have a M1 Max 64GB over a M5 Pro 64GB.

At this point, I really want all AI benchmarks to use the best coding/writing model that normal people can run locally.

For now that seems to be Qwen 3.8 27B that requires ~28GB of RAM (Q4) which is pretty doable for anyone with a beefy GPU or slightly-above-average mac.

I want to know how close open weights (open models) are to all these leading commercial models. Yes, yes, local models aren't as good, but that distance is closing as local models are already better than what we had just over a year ago in many cases.

Same with Text-to-Image generation (Flux 2, Z-Image-Turbo, etc..) or text-to-speech (Parakeet, Whisper, etc..).

That reminds me. Can we standardize cypress tests to load the website via a restricted speed as one of the tests? Some of of our modern webpages are still slow on the latest iPhone/Pixel/Samsung over 5G.

8,600,743 bytes of assets and frameworks later I'm finally able to see your contact-us address.

I'm not sure about a return to keyboard, it's not large enough to actually type on, and swipe seems to be a much faster alternative to pecking at the keys.

Maybe swipe typing isn't as popular as I first thought.

Rust just makes you sign a waiver first.

Nice, which model quantization is this? Is it on huggingface?

I leave it running at night. No danger of burning my token subscriptions and it has hours and hours to run slowly with a manager like: github.com/kunchenguid/gnhf

Yes, 15-30 t/sec is pretty slow for local models so I recommend running local LLM tasks overnight where (vs paid plans) there isn't a risk of chewing through your token budget from a rogue loop or sub-agent. Even if it takes hours, you're sleeping anyway so no concern. herdr + pi works great for this but there are lots of harnesses.

You are mistaken. I'm running Qwen 3.7 28B 4bit (MLX) with a 200k context window and everything total is 32GB RSS.

Is this the best? No. That's why I said the sweet spot. Getting from 16GB macs to 32GB is perhaps possible. Jumping to 64GB or 128GB as the default is simply unreasonable right now.

If I had to pick a product, I'd say an affordable 32GB mac would be the sweet spot for running local models that function well like Qwen 3.8.

It's true, most people don't run models, but being the default platform for running open weights seems like it has plenty of advantages right now. Just like sales benefited from developers defaulting to MacOS for most open source languages like Ruby, Go, Rust, and TypeScript.

103KB is the actual source code. You should not be reading or debugging the minimized or gzip/transport bytes.