whoami4041

Funny story to build on this. I spent a couple of days last week building a Powershell script for database refreshes (backups + restores). The process was iterative (typical), and AI helped me solve all the problems we've ran into historically here. It helped me write tests for all the gotchas I had stored in my head from watching various approaches fail over the last couple of years. I ended up with pre and post hooks on every side, and at a high level I'd vastly improved the time to delivery because (as I initially stated) I knew EXACTLY which problems I needed to solve there, and AI helped me get there in a fraction of the time. The end result was a common .psm1 file, 2 ps1 scripts (the orchestrator and executor script), and a total of about 2k lines of Powershell, which I hadn't combed through line-by-line because the tests were solid and I tested it in non-prod prior to releasing to production.

The first run in production ended up causing an unhealthy db in RECOVERING state because there was a tiny logic bug where the restored db wasn't verified as ONLINE before it set it MULTI-USER (which our db state monitors caught within the couple seconds of that being true).

I think AI has enabled us to solve problems much quicker without being intimate with every tiny detail of the code itself. The funny thing about the story is that this isn't a novel problem. Coming in behind a peer who wrote the thing, or even coming in to debug a script I'd written myself a year ago is the same. AI has just pushed up the time to lacking knowledge of precise details of logic, so you can ship something and be unaware of EXACTLY what it does the same day, instead of 6+ months later.

I'm not sure if this is right or wrong, but debugging it was 3 minutes of presenting the issue back to AI instead of 2 days of combing through 2k lines of Powershell that I may or may not have written.

IMO we have been conditioned to believe that everything is "all or nothing" which is a mistake. AI-assistance provides undeniable leverage IF, and only if, you know what you want. I think the market, in general, is still finding where things level out here. The problem, which I have been guilty of myself at times, is offloading key decisions. Humans absolutely have to retain those. AI can be used to present options, but the accountability always rests with the human (at least for now).

I actually hold both extremes inside of me simultaneously. The speed at which you can ship when you have a strong vision of the end product and the architecture is extraordinary (the part of me that loves AI-assistance).

The journey itself, at least for me, has been absolutely grueling though; I'd say ~30% of the time it's just straight up soul-sucking. Some of that could be because of my own incessant need for discipline and clean code. I don't know how people let agents run wild in hours-long workflows, I can't even get Opus to stop running my test suite repeatedly to look for failures even though cargo test fails fast (the model already knows this), CLAUDE.md has the exact steps and commands for running the test suite, every invoked skill explains the same, and the hook rejects repeated attempts with the same explanation. It STILL, 90% of the time, uses whatever command it wants, bypasses the hook's cooldown to try a different grep because its own invented command didn't return any failures, and if it doesn't bypass the command it tries to wait it out so it can try again. Such a simple thing that it can't get right no matter what I've tried.

Anyways.. Love the leverage, hate fighting with the model on the way from A to B. Everything it does should be challenged.

People that "hate" AI are either expecting it to do too much and are disappointed or aren't watching it closely enough and have to suffer through refactors after they thought they'd been making progress. People that are only over the moon may be working in less complex systems and haven't felt the pain of all the failure modes yet, or just aren't yet aware of the bugs hiding in what AI produced.

Anyone who has built something significantly complex enough probably shares the same love/hate relationship.

I totally agree. However, none of them are infallible and never will be. They're nondeterministic by nature. There is an interesting psychological nuance that I've noticed even in myself that comes with AI assistance in coding, and that's the review/approval fatigue. The model could be chugging along happily for hours and make a sudden, terrific error in the 10th hour after you've been staring at reasoning and logs endlessly. The risk of missing the terrific error in that moment is very high at the tail end of the session. The point I was making (poorly) is that in this specific domain, where businesses are making data-driven decisions on output and insights that can determine the trajectory of the entire organization, human involvement is more critical than, say, writing something like a python function with an LLM.

The key to my point is in the word "generating". Meaning human input/judgement by actually typing more SQL than the LLM produces. The model's reasoning and code generation pipelines are typically 2 separate code paths, so it may not always actually do what it intends which can lead to unexpected results.

Very interesting work here, no doubt. It's a measured approach to using an LLM with SQL rather than trying to make it responsible for everything end-to-end.

"LLMs are good at SQL" is quite the assertion. My experience with LLM generated SQL in OLTP and OLAP platforms has been a mixed bag. IMO analytics/SQL will always be a space that needs a significant weight of human input and judgement in generating. Probably always will be due to the critical business decisions that can be made from the insights.

That's a pretty lofty valuation for a company that has yet to demonstrate code generation anywhere near Anthropic's models if they're leaning into the engineering angle.

Playground is not mobile friendly. Lexega is a pre-execution analysis and policy enforcement engine for SQL in CI/CD and agent runtimes

Lexega is a pre-execution analysis and policy enforcement engine for SQL in CI/CD and agent runtimes