ekojs

Tried it out, asking it for a broad suite of real dev work:

- UI component polish on a React FE

- General review of a complex cross-cutting Node backend changes

- Minor changes to a Rust server with reviews by Codex

Well, my first impression is that it's still a step below Astra/Opus5.5/Fable here. Intelligence is there I think, but 'taste' and 'judgment' is not there yet. Personality seems okay but that doesn't help when it got stuck into a loop by being fixated on the review comments (on iteration 7 of review->fix last I checked) without taking a step back and reconsidering, and this is on a relatively simple change.

Limits seems okay-ish on the $50 plan but not too generous I think.

Yeah, bad wording it seems. Though a charitable interpretation is that Fable 5 and GPT 5.6 Sol are joint 1st place in the measurement.

> In our evaluations, Kimi K3 delivers frontier-level performance. Among the models tested, its overall intelligence ranks second only to Claude Fable 5 and GPT-5.6 Sol. For the complete benchmark results, see our tech blog. The full model weights of Kimi K3 will be released in the coming days. More details on the architecture, training, and evaluation will be published together with the Kimi K3 technical report.

> K3 pushes the boundary of end-to-end knowledge work. On the GDPval-AA v2 leaderboard, Kimi K3 scores 1687. The benchmark evaluates AI models on real-world tasks across 44 occupations and 9 major industries; Kimi K3 ranks behind only Claude Fable 5 Max and GPT-5.6 Sol Max, and ahead of Claude Opus 4.8 Max at 1600.

> On AA-Briefcase, Kimi K3 scores 1527, ranking second among all models — behind only Claude Fable 5 Max and ahead of GPT-5.6 Sol Max (1495). AA-Briefcase is a private agentic knowledge-work benchmark developed by Artificial Analysis to evaluate frontier agentic capability in long-horizon knowledge work.

Really good benchmark score it seems. Maybe another DeepSeek moment right here.

Seems like the only good thing about 3.5 Flash is its speed. Not cost-competitive or benchmark-leading by any means.

Not disagreeing with your argument, but:

> If you want a good dense model, use qwen3.6 27B instead, speed will be up, and if you don't take my word for it being smarter, take openrouter's prices of it against the bigger, slower and less memory-efficient gemma do the talking.

Don't know if this is the correct read. I think those providers are simply taking cue from Alibaba's first-party pricing for the 27B Dense. It's kinda overpriced imo. Perhaps it can be explained by how 'reasoning-inefficient' (relative to frontier models or even Gemma) the Qwen models are and longer sequence lengths are expensive to serve.

> HTTP is just not a good transport for streaming LLM tokens and for building async agentic applications

I don't know if I agree if this is a problem with SSE or HTTP. Something like a Redis Streams-backed SSE would solve most of the 'challenges' presented in the post.

> You cannot run these models at 8-bit on a 32GB card because you need space for context

You probably can actually. Not saying that it would be ideal but it can fit entirely in VRAM (if you make sure to quantize the attention layers). KV cache quantization and not loading the vision tower would help quite a bit. Not ideal for long context, but it should be very much possible.

I addressed the lossless claim in another reply but I guess it really depends on what the model is used for. For my usecases, it's nearly lossless I'd say.

Yeah, figure the 'nearly lossless' claim is the most controversial thing. But in my defense, ~97% recovery in benchmarks is what I consider 'nearly lossless'. When quantized with calibration data for a specialized domain, the difference in my internal benchmark is pretty much indistinguishable. But for agentic work, 4-bit quants can indeed fall a bit short in long-context usecase, especially if you quantize the attention layers.

Not at all, I actually run ~30B dense models for production and have tested out 5090/3090 for that. There are gotchas of course, but the speed/quality claims should be roughly there.

As this is a dense model and it's pretty sizable, 4-bit quantization can be nearly lossless. With that, you can run this on a 3090/4090/5090. You can probably even go FP8 with 5090 (though there will be tradeoffs). Probably ~70 tok/s on a 5090 and roughly half that on a 4090/3090. With speculative decoding, you can get even faster (2-3x I'd say). Pretty amazing what you can get locally.