jmaker

Since that’s “likely” correlated (and a “likely” is often enough), a product manager will take the most optimal decision for themselves and the company - reject such use cases as unsupported. This is easier than to manage such an edge case for “a few” legitimate users.

Few orgs understand compliance. While there might be some compliance people at the org level, individual teams implementing features will need to have a good line to them. In the end, knowledge is siloed and everyone needs to deliver on time. So it’s easier to forbid with some controlled collateral damage than to permit with an uncontrolled downside variance. Just a derisking hedge.

Serving use cases that drift away a bit “too far away” from the average (+ some sigmas) use case is not economically justified for a product manager - their KPIs would deteriorate from such “altruism”. No one will care if a few low-margin customers can’t use the product.

I couldn’t find a way to decline model training and reduce data retention for antigravity or Gemini. Apparently it’s only available on a business/enterprise plan, not personal. Did you manage to solve it? That’s the only reason I don’t use Gemini or antigravity.

This is an interesting hypothesis to test. Depending on the embedding model, such less frequently encountered character sequences in the coding domain could indeed produce less optimal tokens. A dedicated model trained from scratch on Haskell would be optimal but wouldn’t generalize well beyond that, but can serve as a deliberately overfitted baseline.

What I meant was really about the output tokens, on the API pricing level of interaction with an LLM.

I’m really happy to hear that. I wish Haskell were more widely adopted and more generally accepted as a solid enterprise choice without the bus factor. I can confirm that the applicants are usually great in every respect. I tried to introduce Haskell in multiple enterprises but failed at engineering managers’ reluctance to admit Haskell as an option - every time because of the bus factor argument. Scala used to be easy to advocate for as an alternative. To me, with LLMs there’s really no need to care about the choice of the runtime language unless there are SLOs to meet or library choices due to domain peculiarities.

As for token-wise inefficiency, this is based on my sporadic observations and discussions with friends and colleagues from the past three years. I can’t give you a fresh benchmark with latest models in a reproducible way. But I’m happily willing to accept your assertion at face value. This makes me curious to see for myself how the latest models perform, will perhaps set up a quick evals just out of curiosity.

From my past experience, the latest I’ve seen were Opus-4.6 and GPT-5.5 struggle a lot with standard GHC Haskell, no extensions, nothing fancy. Not that they produced impeccable Python or Rust. But it appeared to take several turns for obvious expressions, while at Java and TypeScript they were much better - fewer turns, time, and cost for the same verified results. I had assumed that the training set wasn’t large and diverse enough for Haskell and Scala, and this was the go-to explanation with everyone I talked about it. Generated F# code was good enough but not OCaml. With C++ there’s still quite some struggle, expectedly.

To me it resorted to the question, if we’re mostly generating code through LLMs now, which ones and what will it cost in total terms per task/capability completed. I’m wondering now if DeepSeek/MiMo or GPT-6 Luna can benefit from the terser, more (forgive the pun) load-bearing expressions.

I still love Haskell, had dedicated several years to it while at the university. Was impossible to land a job with Haskell. Did some Scala but it’s in demise. F# and OCaml are extremely niche. Ever fewer opportunities and engineering management is convinced it’s impossible to hire functional programmers.

In this new AI-driven world, is there still place for such a luxury as functional programming?

I mean few people still code by hand, few read the generated code, models aren’t trained on functional languages, it’s inefficient token wise to use functional languages - while a lot become self-proclaimed software engineers overnight by just prompting LLMs.

Opposite experience here. SuperGrok allowance is drained extremely fast. About 20-30 simple coding or code review prompts in grok build, and it’s over for the week. When using it via the Grok app, it can last longer, unless you need to edit photos. I didn’t bother to properly measure grok’s burn rate on the SuperGrok plans. It’s just not there yet.

What really is fun is when the opposite of the public outcry about purported distillation happens - when Astra in Codex suddenly responds in Chinese. Now that is fun. Wonder what it’s routing to.

For me Sol is less efficient than Astra - Sol makes many avoidable mistakes and has issues with context compaction - sometimes it goes haywire after a couple compactions.

Agreed on the “being dumbed down” observation. It appears they’re most powerful at release time and then are gradually “optimized” so every new model feels more powerful. But there’s no evidence on routing to a deployment with other weights. It would be plausible to do so though at least at peak times.

I have done the same. I hesitated for way too long. I shouldn’t have.

I get way more usage for way less money without any quality or performance degradation. My $200 Codex Pro plan allowance is depleted in 2-3 days. Sometimes Tibo announces a usage reset. But GPT-5.6 models are really not good for coding. Sol has been making increasingly more mistakes in the past two weeks even in the reviewer and advisor roles. Astra is usable for coding but slow and very expensive. In the past two days I’ve used up over 70% on simple copy editing, with dedicated short specs and short sessions. Really little one can do to make it more efficient. Similar work took 20% at most just a month ago. I’m looking to use Astra for milestone reviews/advisory. Perhaps a $100 Pro downgrade will be enough. But my main work is now on open-weight models. And you don’t need to depend on someone to send you a reset. And it’s cheaper by the end of the month too.

With Claude the limits are not even fun anymore - my weekly $100 Max plan quota is gone in one day on merely review invocations, no coding. And my $200 Pro quota is gone in two with some coding. Sonnet 5 is not usable for coding. And Opus 5 tends to always make a couple avoidable mistakes on every task. Fable 5.1 is ok but tends to ignore skills and to work around explicit instructions. Completely canceled all my Claude subscriptions.

With Qwen 3.8, DeepSeek 4.1 Flash, GLM 5.3 I’ve been getting Opus 5-level performance, with less blah blah and no overengineered churn. Public benchmarks are really not telling the real story. The models are more dependable and more predictable. They have their own failure modes. Sometimes DeepSeek 4.1 Flash is quite stubborn but it fails in a good way. Bad for full autonomy - I need to intervene, but it sticks to the rails and instructions - other than Fable and Opus that try to outsmart you and your harness.

Grok is interesting but has been a bit underwhelming on Grok plans - my SuperGrok allowance is depleted in a single session overnight. SuperGrok+ gives more but it’s still about the same as with OpenAI, Claude is way less now.

Since the allowance volume has been shrinking with the major model providers, to me, open-weight alternatives are really necessary now to at least maintain the momentum and budget.

But at work it’s really an uphill challenge - it’s become impossible to convince the tech leadership once they got hooked on Anthropic. They No facts will help. Some people underestimate how expensive Claude really is after getting used to the subscription plans with allowance resets. OpenAI models are expensive too.

Why not the other way around?

I don’t get Claude, and that’s almost exactly what I did - I dropped to Claude Pro $20 + Codex Pro $100, and then unsubscribed from Claude and ramped up Codex. The Claude Pro is consumed within an hour on a simple task. I wish only Codex worked a bit faster than on the Fast mode.

I used to rely on Fable for research when it was first out, today it doesn’t seem to be much better than Opus, and it uses up the quota exceptionally fast - 1h Fable in a single short session, and there’s little left for Opus to hit the 5h limit in a second session. With Opus I get about 3-5h of relaxed use with a couple subagents to save the context, but there’s usually quite some disagreement between the subagents and orchestrator - Claude does some model routing with default agents and picks Haiku and Sonnet for subtasks - only later to disagree with them and redo the work - and burn extra tokens. With Claude, it’s really either Opus or Fable if you want some quality.

That said, their marketing is exceptionally effective. Virtually all nontech folks consider only Claude.

I don’t get how Claude is considered providing “unlimited” quotas. I use up my 5h on Max $100 and Team Premium in 2-3h of relaxed use of Opus 5 high+ on fresh sessions with just a couple skills/plugins. And my weekly quotas are gone in 3 days of such relaxed use. With Codex my $100 weekly quota is used up within 3 days with Sol high+ too.

Im not convinced to pay $200 for Claude’s models.

With Claude, I have to intervene every 15-20 minutes, it’s non-autonomous and it’s incredibly unreliable at self-correction. GPT is strong at self-correction but it tends to drift away from the plan to self-correct in a loop very often - a lot of tokens and time burnt on aimless churn. Opus tends to push its uninformed opinions and fake retrieval, drifting every turn increasingly farther from the intended and approved design. Opus skims over specs and makes too many mistakes.

As for closed frontier models, I prefer the GPT models over Claude’s.

I’ve started relying more on Grok, GLM, Kimi and DeepSeek models for subagents - I’ve ended up with a factory and am seeking to reduce my reliance on the closed frontier models - they’re just not SoTA on their own for development anymore.

Cue back in ten years and we all talk like that.

Whatever I used Sonnet 4.6 for, including Claude Code and Claude Chat, it made so many mistakes and totally awkward assumptions that I can’t fathom what it’s supposed to be good at. The mistakes were so blatant. Plan mode, several passes, couple grand in API costs… just disappointing at every task in every session over the past few weeks. Opus 4.6 has been good, still quite a few unexpected, silly mistakes, a few subtle but critical mistakes, but produced workable increments and code reviews, vastly subpar to GPT-5.x in chat mode (with and without identical customization).

Great point. What I’m recognizing in that PR thread is that the bot is trying to mimic something that’s become quite widespread just recently - ostensibly humans leveraging LLMs to create PRs in important repos where they asserted exaggerated deficiencies and attributed the “discovery” and the “fix” to themselves.

It was discussed on HN a couple months ago. That one guy then went on Twitter to boast about his “high-impact PR”.

Now that impact farming approach has been mimicked / automated.