ZephyrBlu
Joined 7 years ago
Joined 7 years ago
Astra wasn't even released 3mo ago. It would not be surprising in the slightest that a public model from 3mo would not be capable of solving this problem, but an internal one from current day would be.
I found the Devin reviewer to be very good, and have heard good things about Cursor's Bugbot. I've also found asking an agent with fresh context or subagent to adversarially review locally is good.
I don't think that is what most people colloquially mean by "feature flags" though. Even most teams in Shopify abused "ephemeral" flags for long periods of time.
When they rolled out the mandate it was very annoying for my team because we had a lot of operational flags like you're describing that we needed to get exemptions for.
When someone else flips a flag that impacts your team and they have no idea they even caused a problem, it becomes very difficult to resolve the issue. Usually you can check for recent deploys, instead you have to go and guess at which feature flag which was recently flipped could possibly be affecting your code. I experienced this several times.
Also, it was actually more desirable for most of these things to go straight to production. Test it properly before shipping, then when you ship it soaks on a 5% traffic canary at which point you can monitor and cancel the deploy if you see errors. That is generally safer than a feature flag rollout unless you are doing something very high impact/risk, in large part because it gives any other team affected by your rollout the ability to respond and be able to easily find the source of errors.
In my org it was a fairly common failure mode to ship something and accidentally cause an issue for another team. Usually it was other teams/orgs shipping things that impacted us.
If it's config/customization, it should be in code. If it's experimental it can be a flag until it solidifies, and then it needs to get moved to code.
When I was at Shopify a couple of years ago they mandated that feature flags had to be short-lived (Like 2-4w lifetime tops, some had exceptions) because they would end up getting left in code and never cleaned up, or for extended periods of time like months. Hard to tell if it's genuinely a "feature flag" or actually just a normal part of the system at that point.
Feature flags being flipped in prod was also a major source of incidents, in part because people didn't treat them as experimental and with the associated risk profile of something experimental.
The only exception where having long-lived flags was useful and required was for operational killswitches (E.g. disable Apple Pay because it's having issues), but that is explicitly not application config.
Microsoft's previous model was not linked to tokens at all. Complete anomaly among coding agent providers. It's not representative of token economics at large. Claude Code recently announced increased limits. Codex does regular limit refreshes.
Tokens are pretty damn abundant even though they're not bargain basement cheap yet.
Previously they used "premium requests" which would allow you to make a request to one of the more expensive models. People abused the shit out of this because a request was disconnected from tokens.
You could make one request which used tens of dollars worth of tokens, obviously not the intended usage pattern and obviously unsustainable.
Tokens for a given intelligence level are becoming much cheaper very quickly, but everyone wants to use the smartest frontier models so tokens are not dirt cheap. Even frontier models are a bit cheaper in absolute terms than they previously were, and much cheaper in terms of intelligence.
It's also absurd to think that there is no company which does genuine software "engineering". If you break ads at Google/Meta, streaming at Netflix, etc there are massive consequences. They are heavily incentivized to properly engineer their systems.
The main thing that governs whether time is spent to well-engineer something is if there is incentive to do it. In traditional engineering that incentive is the law (Getting council approval, not getting sued, etc). In software engineering that incentive is revenue.
The "it saves money" mindset is bad for everyone, distorts everything away from valuable work and investments that generate the most revenue.