rednb
Joined 11 months ago
Joined 11 months ago
However where i think you have a point is that it tends to run tests quite casually where a reviewer would have been enough to catch mistakes.
IMHO, the real problem is the 5min cache way too small. They should up it to 1 hour.
Buid alone takes 4 minutes. Test suite heavily parallelized runs in about 2 to 3 minutes, infrastructure dependencies fully containerized using podman, no external API call.
And this is without mentioning the 5000 frontend tests (angular), including Playwright end-to-end tests wired to the backend (about 3 to 4 minutes run without the e2e tests).
And this is running on a dev machine with 64gb of RAM and an NVMe disk.
Some systems are just massive, and even if you run focused tests, it's easy for a change to have a large blast at least requiring a large rebuild, and at times to rerun a larger set of tests.
In my opinion sunsetting JetBrains Space was a major strategic mistake. It would have given them the moat they currently crave for to compete in the AI space, by enabling cool AI powered features (codex like sandboxes connected to their ides etc...).
All their AI products currently feel like desparate shots in the dark, while Jetbrains Space would have been a natural home, delivering strong value and differentiation.
Once Claude Code, Codex and OpenCode matured and discredited Junie, the next thing i looked for was a code review and CI/CD environment that was as seamlessly integrated to my IDE as possible. I was baffled to discover that JetBrains had just that but decided to sunset it in the midst of the AI war. Absolutely baffling....
The fact is, most people aren't builders and never will be. Building software is about formalizing and automating processes, and 95% of folks aren't interested in putting even the most basic effort into improving their business by approching it at a strategic level. Too focused on keeping the lights on and/or more interested in field operations.
For another example look at the many businesses using @gmail addresses to run their business while Google Workplace has streamlined custom domains for a while.
So in practice, ChatGPT Sites will mainly enable software agencies to build websites and offload maintenance to the customer : just manage it from your ChatGPT account. The equivalent of Bring your own cloud for Mom and Pop businesses, and other small companies. Not even sure ChatGPT Sites will create meaningful downward pressure on agency rates.
Lesson learned the hard way.
Except maybe that the model often believes that he is running out of context, and needs to rush so i occasionally need to jump in to tell it that it still has plenty of room left.
But this does not degrade the quality of my overall experience in a meaningful way.
This is definitely not about dealing with the frontier of AI. I wasn't part of the nerfing chord, but Astra changed my mind. Quality got me to upgrade from Pro x5 to Pro x20 on launch day. A couple of days later was dumb af, horrendous code quality etc...
Something fishy, or at least unethical is going on. Not sure it impacts API users though.
Deepseek 4.1 ranks very low in this benchmark but it has proven so capable that after being simultaneously on Max x20 and Pro x20 subscriptions, i've transitioned to using DS 4.1 as a daily driver and am very satisfied.
My point is, i think their overall ranking makes sense, matches my experience with out of the box capabilities for vague and underspecified tasks. But seeing a model rank low in their ranking does not mean that the model is incapable. Having skills and guidelines has a lot of influence on what you get out of a model.
Not criticizing because i can see that many people love Apple products precisely for this reason. Apparently this makes the UX friendly and intuitive. But it didn't click for me.
So it is very unlikely that a neighbor will have the audacity to attack them, and even if this happens, i'd expect the war to be over rapidly. Just like if Canada or Mexico tried to attack the US.
As a Max x20 and Pro x20 subscriber, can tell you that it doesn't matter since they continually move the baseline of token use So in practice you feel that you're continually getting less from your subscription.
While it never happened to me in the past, i reached my weekly limit within 3 days using Opus 5. And the Open AI weekly limit essentially is a Claude Max x20 5-hour limit. Not even talking about the baseline in intelligence : on release day Astra was so good that it lead me to move to Pro x20. Now it's dumb af and token use is insane.
Deepseek 4.1 Flash has been a lifeboat for me, finally able to work without being constrained/distracted by limits and with what is in my view even better intelligence than Opus 5 for a fraction of the costs. DS is not messing up my brain with load-bearing pseudo jargon in every sentence. It respects coding guidelines, and completes even the most complex tasks most of the time in one shot.
DS 4.1 had been able to add complex features to my repo without breaking a sweat (330k lines of F# + 4M circa lines of an Angular frontend). Writes very idiomatic F# and respects our guidelines and style perfectly. Just completed an extensive UI/UX research and implementation work.
I am ditching both x20 subs and will only keep a Pro x5 because wife does a lot of design work and needs solid image generation capabilities.