namnnumbr
Joined 5 years ago
Karma
290
Submissions
33
Member since
Nov 16
Recent submissions
Comment · 3 months ago
That's a misleading title - of course no existing MCP server is going to be compliant with a spec that has not yet been released, and obviously no server is going to incorporate substantial, breaking changes when the clients themselves don't support the spec...
Comment · 4 months ago
oMLX (https://github.com/jundot/omlx) makes running the mlx inference server quite easy for those interested in UI-based hosting. oMLX also supports mtp or dflash drafting.
Comment · 5 months ago
Yep. or https://stopsloppypasta.ai
Comment · 5 months ago
stopsloppypasta.ai
Comment · 5 months ago
AFAIK "orbital data centers" are a bunch of nonsense.
1. GPUs create heat. There's no efficient way to get rid of the heat in space (vacuum is an insulator). 2. Die-shrink makes modern processors and memory more and more susceptible to radiation; shielding is possible, but adds cost + mass (which adds cost)
Comment · 6 months ago
I really like latent.space and simonwillison.com.
Also (shameless self-promo) I publish a 2x weekly blog just to force myself to keep up: https://aimlbling-about.ninerealmlabs.com/treadmill/
Comment · 6 months ago
Yes! I'd be totally happy with today's sonnet 4.6 if I could run it locally.
If you can forgive the obviously-AI-generated writing, [CPUs Aren't Dead](https://seqpu.com/CPUsArentDead) makes an interesting point on AI progress: Google's latest, smallest Gemma model (Gemma 4 E2B), which can run on a cell phone, outperforms GPT-3.5-turbo. Granted, this factoid is based on `MT-Bench` performance, a benchmark from 2023 which I assume to be both fully saturated and leaked into the training data for modern LLMs. However, cross-referencing [Artificial Analysis' Intelligence Index](https://artificialanalysis.ai/models?models=gemma-4-e2b-non-...) suggests that indeed the latest 2B open-weights models are capable of matching or beating 175B models from 3-4 years ago. Perhaps more impressive, [Gemma 4 E4B matches or beats GPT-4o](https://artificialanalysis.ai/models?models=gemma-4-e4b%2Cge...) on many benchmarks.
If this trend continues, perhaps we'll have the capabilities of today's best models available to reasonably run on our laptops!
Comment · 6 months ago
IMO, you're incorrect:
1. In my own use, since 1 Apr this month, very heavy coding:
> 472.8K Input Tokens +299.3M cached > 2.2M Output Tokens
My workloads generate ~5x more output than input, and output tokens cost 5x more per token... output dominates my bill at roughly 25x the cost of input. (Even more so when you consider cache hits!) If Opus 4.7 was more efficient with reasoning (and thus output), I'd likely save considerable money (were I paying per-token).
2. Anthropic's benchmarks DO show strictly-better (granted they are Anthropic's benchmarks, so salt may be needed) https://www.anthropic.com/_next/image?url=https%3A%2F%2Fwww-...
Comment · 6 months ago
Following up on "strictly better" via plot in release announcement:
https://www.anthropic.com/_next/image?url=https%3A%2F%2Fwww-...
Comment · 6 months ago
The title is a misdirection. The token counts may be higher, but the cost-per-task may not be for a given intelligence level. Need to wait to see Artificial Analysis' Intelligence Index run for this, or some other independent per-task cost analysis.
The final calculation assumes that Opus 4.7 uses the exact same trajectory + reasoning output as Opus 4.6. I have not verified, but I assume it not to be the case, given that Opus 4.7 on Low thinking is strictly better than Opus 4.6 on Medium, etc., etc.
Comment · 7 months ago
if you used an AI, I'd love to see the prompts you used to get such human grammar and spelling errors
Comment · 7 months ago
I talked myself in circles on that "why it matters" heading but ultimately couldn't come up with a better one. "The problem" has similar ai-slop feel, and "the rant" // "the rules" didn't really evoke the feeling I wanted.
Happy to take suggestions on this!
Comment · 7 months ago
Oh, I 100% acknowledge the site itself was LLM generated. I'm not a web designer, so I needed a lot of help making a visually appealing site, even if that design language is at this point LLM trope.
However, the essay and the guidelines were all human-written!