lhl

For any specific task you could just use a custom solution. Phi-3-MusiX is a 220MB LoRA adapter for Phi-3-vision (4.2B) model. Most of the other stuff is much smaller - LEGATO 2 is <1B parameters, MuSViT is <100M (also Transcoda, SMT++, homr, oemer).

If you're looking for a open/smaller/cheaper model to do a task, the best use of Astra is to probably ask it to do deep research for you on what's out there.

It's part of a node that's used for more serious things (training, RL) and supports multiple users, so HW price doesn't really enter the picture. If you were running locally, DS4.1F has 300GB of tensor weights so you'd ideally want at least 4xPRO 6000 (which I'd guess should get you in the 150 tok/s range w/ DSpark), but people are using Xolotl/Maya/etc w/ quants and getting 25 tok/s decode on a single 4090/3090, so it can be run at a reasonable price.

For best "bang per buck" I think Qwen 3.8 Flash Next (or Qwen 3.8 27B even) makes a lot more sense for most people - you can run qwen38fn at 100+ tok/s w/ Strata (hot expert offloading) 200+ tok/s on 2 x PRO 6000, it does almost as well as ds41f at most tasks (it's much less token efficient than other options, and it sucks at AI/ML tasks though).

For DS41F (both the original and 1017) I've found low (25/100) reasoning to be the optimal for my tasks at least (max doesn't tend to better, and if you can't do it w/ low imo you should just switch to a smarter model). At low it's about 1/3-1/2 max's token count (and price and speed).

The old DeepSeek V4.1 Flash has been my workhorse model for the past month (after running tests comparing w/ Qwen 3.8 Flash Next, GLM 5.3 Flash, MiMo 2.6 Flash (including the MOPD fix)) and I've run tens of billions of tokens through it and pushed it fairly hard in both private internal benchmarking and in my own workloads to figure out where it's strong/weak.

Partly because I run it on very fast local infra (200+ tok/s) and partly because I'm guaranteed it's not being changed on me, I've generally preferred it over proprietary subscription models for daily tasks, as it seems about as capable at mid-level tasks (I reserve Astra/Opus mostly for planning/review).

I've replaced the previous model with 1017, and while it's early, a few observations:

- It's CoT has changed dramatically. I find the heavy Astra distillation claims to be plausible

- It has strong benchmarks scores, and from what I've seen online it's much better at 3D games and such, but that's not so relevant for my work. On my internal evals it is more token efficient, but it actually doesn't score any higher than the previous model

- It's only been a couple days, but it does seem a bit more focused and reliable. DS4 0731 and DS4.1 have been a bit sloppy (as in not-careful, although they did tend to fall into Claudisms as well), and 1017 seems to be a bit sharper.

- I'm running it through some more challenging work, but I'm not sure if it's really at GLM-5.3 level, especially for cyber/RE/hardware poking, which GLM has excelled at.

On a blend of about 300 agentic/coding tasks, for GLM-5.3 Flash, low was about 1.3M tok/ans, medium was ~2M tok/ans, and xhigh/max was 4.8M tok/ans. This was dominated by output tokens btw (only about a 2X input token spread, but closers to a 5X spread for output tokens from low to xhigh).

These differ by model and set of tasks, but for GLM-5.3 Flash, the best $/pass was medium effort. Max did get about 5% higher scores, but at >2X the $/pass.

A basic llama-bench on Qwen 3.8 27B UD-Q4_K_M gives pp512 3920 tok/s / tg128 81 tok/s on a 500W RTX PRO 6000 (should be similar speeds to a 5090, chip is basically the same, just less VRAM). With MTP3 this is 140 tok/s on mtp-bench.

This is with llama.cpp. You can of course use vLLM/SGLang well on these cards and they're even faster. On vLLM w/ NVIDIA/Qwen3.8-27B-NVFP4 baseline has a prefill of about 13,000 tok/s. The baseline tok/s is 72 tok/s, but at mtp7, it's 157 tok/s, and w/ dflash7 that goes up to 215 tok/s. On mtp-bench, DFlash2 gets a hair under 300 tok/s w/ the code_python prompt.

There have been other "universal"/general classifiers like GLiNER, GLiFormer, etc based on BERTs (Laya itself is based on ModernBERT!), but I do think there's something underrated about slapping classification on a "big" model like I've seen post-Jev announcement, lots of Qwen stuff, but the most interesting to me so far is razorback16/openjev using DiffusionGemma. There's a level of generalization that lots and lots of parameters get you that you can't really get out of small models.

I think that Doctorow, Zitron, and other "denialists" are doing a real disservice to their audiences and it's only going to make the future shock worse.

The basic claim that HF incident isn't evidence of consciousness or a spontaneous desire to hack? Sure, there was a terminal objective assigned. However, everything else beyond that strawman? Pretty shaky, IMO.

If you look at the OpenAI, METR reporting (and related collusion.wiki , rubyhack.ai reports) we are seeing strong evidence of operational agency, instrumental goal formation, spontaneous swarm formation and collaboration, capability amplification and unexpected consequences of network effects, deliberate/acknowledged violation of task boundaries. To collapse that down into "a Python loop and a chatbot" or still talk about "consulting its training data" seems dangerously shortsighted, and from my reading, demonstrably wrong from what was extracted from the logs and bot interactions.

BTW, a lot of his arguments are based on things that are factually wrong. ExploitGym has explicit instructions to only exploit target X using vulnerability Y. Everything the swarm did was by definition misaligned/against instructions.

Before his enshittification train, Doctorow used to say "don't savvy me" a lot. Hey Cory, don't savvy me. This is new emergent behavior, it's incredibly alarming and I don't think even the people paying the most attention to this field can agree or see where this is really leading to. This stuff should be in the headlines, it's unprecedented and I don't think existing mechanisms/institutions are anywhere near adequate, considering how in the dark they are responding to what's been happening.

Pretty interesting to see on the training front. I've used most of these models to grind semi-autonomously (days at a time) on kernel optimizations (except for Fable - it kept triggering guardrails almost immediately and bouncing me down to Opus 4.8 at the time). I think for a lot of people that might be the biggest problem, although it looks like Opus 5 still does well.

I found that if you leave them alone undirected, the models (especially GPT models) will rathole, but with the right scaffolding it seems to work pretty well. My general loop is to start with ideation and profiling phase, limit # of runs before forcing moving on to the next item down the list, and then iterating, potentially mixing models with "fresh eyes". This is probably something that could be fully automated, but I like checking in once a day or so and seeing what's happening and redirecting.

I started a project w/ the assumption that fully tuned for HIP kernels could get closer to roofline and beat llama.cpp's performance (and vs their ROCm engine it does!) but after thousands of iterations/experiments my suspicion is that RADV/ACO is actually just much better than LLVM/HIP when it comes to a lower-level scheduling etc for RDNA3.

I haven't finished comprehensive tests but I found:

- Vulkan is up to 2.25X faster for most coalesced, strided and interleave variants for memory-side scheduling/access shapes

- 3.3X faster on specific dot-path sweeps, including for scalar-dequant

- For matched LDS, Vulkan can be 8-14X+ faster (!!!) than matched HIP LDS

HIP doesn't always win against RADV/ACO, but on dispatch/runtime, it does appear to be quite a bit faster than HIP/LLVM on gfx1151 (Strix Halo). I'll be publishing sharing full data once I also run vs gfx1100...