sdrinf

gemma-4 is an incredibly good model, beating several others five time its' size. Hope Jeff's departure won't impact their next-gen products! And best of success for their new ventures!

hi, great project! Windows support is sorely lacking, though. As someone working a lot with sandboxed LLMs right now, the options-space on windows for sandboxing is _extremely lacking_. Any plans to support it?

Just want to echo the recommendation for qwen3.5:9b. This is a smol, thinking, agentic tool-using, text-image multimodal creature, with very good internal chains of thought. CoT can be sometimes excessive, but it leads to very stable decision-making process, even across very large contexts -something we haven't seen models of this size before.

What's also new here, is VRAM-context size trade-off: for 25% of it's attention network, they use the regular KV cache for global coherency, but for 75% they use a new KV cache with linear(!!!!) memory-token-context size expansion! which means, eg ~100K token -> 1.5gb VRAM use -meaning for the first time you can do extremely long conversations / document processing with eg a 3060.

Strong, strong recommend.

Counterpoint to peeps on this thread:

* This approach is the _most consistent_ with retaining anonymity on the internet, while actually helping parents with their issues. If any age-relevant gatekeeping needs to be made on the internet at all, this is the one I find acceptable.

* this is because the act very specifically does NOT require age _verification_ ie using third-parties to verify whether the claimed age is correct. Rather, it is piggybacking on the baked-in assumption, that parents will set up the device for their kids, indicating on first install what the age/DoB is, then handing over the device -a setting which can, presumably, only be modified with parental consent

* yes, there are edge cases, esp in OSS, and yes, it would be nice to iron those out -but the risk = probability x impact calculus on this is very very low.

* If retaining anonymity on the internet is of value to you, don't let the perfect be the enemy of good enough.

Taking the opposite side of that bet, here is why:

* even if an openweight model appears on huggingface today, exceeding SOTA, given my extensive experience with a wide variety of model sizes, I would find it highly surprising the "99% of use cases" could be expressed in <100B model.

* Meanwhile: I pulled claude to look into consumer GPU VRAM growth rates, median consumer VRAM went 1-2GB @ 2015 to ~8GB @ 2026, rougly doubles every 5 years; top-end isn't much better, just ahead 2 cycles.

* Putting aside current ram sourcing issues, it seems very unlikely even high-end prosumers will routinely have >100GB VRAM (=ability to run quantized SOTA 100b model) before ~2035-2040.

I'm working on something like this. Specifically, I'm doing recursive self-improvement via autocatalysis -but predominantly in writing/research / search tasks. It's very early, but shows some very interesting signs.

The purely code part you described is a bit of an "extra steps" -you can just... vscode open target repo, "claude what does this do, how does it do it, spec it out for me" then paste into claude code for your repo "okay claude implement this". This sidesteps the security issue, the deadly trifecta, and the accumulation of unused cruft.

can someone please try running the experiment of "but what if just forking&spinning up an OSS clone, scaling up to take in the migrants, acquire network effects, collect roughly same subscription revenue, but run on just, like, 10 people?"

Discord has a financially and politically vulnerable posture that is downstream of having to operate a very large team, raise funding, be exposed to investor market pressure. However, it is also one of the rare instances of successful consumer freemium subscription monetization. A clone does not have to pay the tuition of "what makes this specific space compelling, and want-to-pay-for"; it just have to _exists_, passively soaking up migrants from each platform shift.

ITT WTB 3rd place for my frens.

Besides the editorial control -which openai openly flagged to want to remain unbiased- there is a deeper issue with ads-based revenue models in AI: that of margins. If you want ads to cover compute & make margins -looking at roughly $50 ARPU at mature FB/GOOG level- you have two levers: sell more advertisement, or offer dumber models.

This is exactly what chatgpt 5 was about. By tweaking both the model selector (thinking/non-thinking), and using a significantly sparser thinking model (capping max spend per conversation turn), they massively controlled costs, but did so at the expense of intelligence, responsiveness, curiosity, skills, and all the things I've valued in O3. This was the point I dumped openai, and went with claude.

This business model issue is a subtle one, but a key reason why advertisement revenue model is not compatible (or competitive!) with "getting the best mental tools" -margin-maximization selects against businesses optimizing for intelligence.

Note: I strongly recommend against using Novita -their main gig is serving quantized versions of the model to offer it for cheaper / at better latency; but if you ran an eval against other providers vs novita, you can spot the quality degradation. This is nowhere marked, or displayed in their offering.

Tolerating this is very bad form from openrouter, as they default-select lowest price -meaning people who just jump into using openrouter and do not know about this fuckery get facepalm'd by perceived model quality.

I actually _like_ this, and so does the comfyweb & weebs who are a very significant portion of the driving force behind calm, decade-long projects.