michaelrpeskin

I did a similar project just for myself recently, and I think AI interpretation of the IRS is probably one of the ideal projects. AI is generally pretty good at following a spec, and the IRS is generally pretty good at writing detailed specs.

In my case, I had an Inherited IRA that, and this is the first year I have to take a distribution from it. It needs to be emptied in 10 years and starting this year I have a minimum amount to take out based on my age. My goal was to write a simulation of how much I take out each year, what that does to my income taxes, how much tax deferred growth I get by leaving it, etc. I have an optimizer to connect to the simulation, so I could then figure out how much I should take each year.

I decided to vibe it, just for fun. I started very simply: write me a simulation for each year with this much projected growth, etc. The AI asked me some questions like "do you have kids going to college during this time frame?" and did you know that your state has an allowance for this much tax free withdrawal, etc. That took me down a rabbit hole since I didn't know all of those things. I said, go read the IRS code and my state tax code, figure out all of the things that are relevant for my finances (normal W-2 employee, no fancy stuff), and it wrote me a much more detailed simulation than I would have thought of and tied it all to all of the tax things that might be relevant for me.

I was shocked at how good of a job it did, I can (and did) independently verify it since my taxes aren't that complicated. It did find savings that I didn't know were available to me.

I think that this is a great area for AI-based development since it's essentially transpiling detailed IRS specs into code, and that's what AI is really good at.

How do these work when plugged in to only one of the split phases on a US-style 120 system? Because I'm a nerd, I know which legs of the phase go to which outlet in my house, but I don't expect everyone to pay attention to that. I assume something has to take that into account somewhere.

I don't think it's as much as opening up what we _could_ do, it's opening up time so that more can be done.

Slop is too generic of a complaint. Do LLMs write code exactly how I would? No. Is my way right? Probably not. But what LLMs are great at are pre-emptively thinking about edge cases that I forget when writing the code. I'm surprised as how many fewer weird bugs come back to me now after a release. Should I have thought of them? Yes probably, but I don't and they do. So in practice, I do think LLMs write better code than me (if you look at results not the actual code itself).

But more importantly is that it allows me to take on projects that I never would have. Some of them are personal projects, some of them are work projects. I often didn't even consider a project because the time it would take _me_ to do it would not be worth the value. But now that _I_ don't have to do it, I can do more things. At the end of the workday last week, I kicked off a project that I've wanted to do for the last 5 years but could never justify writing it. The LLM crunched for 36 hours, I checked in on it this morning and it's ready for me to deploy next week when I get in the office. Something that would have taken me probably 9 months of focused effort. It would never have been worth 9 months of engineering time, but it sure is worth the 36 hours of background work while I was doing something else productive.

I guess it's not a quantum leap in the frontier of what _can_ be done, but it is a leap in what _will_ be done because the cost/benefit parameters change.

I resemble that remark. The trouble is that I'm now too old and if I ever lose my job, I'll never find another one. No one hiring in tech wants the greybeards.

Thanks for the wording on your first sentence there. I've been trying to figure out a way to get that thought expressed succinctly.

I think with AI coding, what we call "good" code changes. Lots of abstractions really only exist to help load the context into the human brain so that they can solve the next problem. If an agent can just search and find all the places to make a change, or to duplicate code with small changes for the next problem, is that bad? Does is just feel bad because that's not what we're used to?

We use structured looping instead of gotos because that makes sense to us, but the compiler still turns it into jumps in assembly. If our interaction is now at a higher layer, do we need good "code" or do we just need good "architecture"?

I don't know, but it's just something I've been thinking about lately.

I said this in another comment a few weeks back - but, at least for me the "joy" of software development was describing what I wanted and getting it. That used to mean fighting with code and trying to be clever and clean or following the right design pattern. But now, I get "joy" from using English to describe my problem and getting the right answer. When the LLM one-shots it, I get the feedback that I'm describing things correctly and clearly. I'm still getting the same joy just in a different form now. I'm old enough and have been doing this long enough that "giving up writing code" is not sad for me. I'm still expressing my ideas, just differently now.

I've also been doing this professionally for about 30 years, plus another 10 as a hobby/learning before then, so I think I have much of the same experience as you, and I do agree. Although, I think with the advent of LLMs, programming is no longer the hard part. The hard part of programming is the convergence of context management for humans while also presenting it to a computer to do something. Much of data structures and algorithms "best practices" are ways to efficiently get your work done as well as keeping it so that a human has context.

For example, one guy I used to work with wrote this really awesome algorithm about 25 years ago, and I'm responsible for maintaining it since he long retired. I can't go into details, but this is the core algorithm in moving billions of dollars between institutions overnight. It's about 10 screenfuls of c that had been converted from the original FORTRAN 77 with dozens of gotos and weird branching statements and about 20 parallel arrays that store indices for pointer chasing. It's almost impossible for a human to follow (I've actually fed it to an LLM and said rewrite this with for loops and no gotos so I can understand it - and it worked!) but it's blindingly fast. The actual problem it solves can be stated in about three sentences, but programming it was hard because of the context management. The reason that my company keeps getting royalties on this is that it's so hard that they'd rather pay us than write it themselves. But I bet an LLM could write it from scratch now.

So maybe programming is no longer the hard part, or at least context management is no longer the hard part and that humans should move up the chain to help manage the context for LLMs so they can get more done efficiently.

Switching topics a little...much of what I think makes coding the hard part was the tension between big-design-up-front and you're-not-going-to-need-it philosophies. Early in my career I worked in health care and that was BDUF and the coding was easy because program managers spent years defining every screen that would be shown to the users, what queries were needed to fill the screen, all of that. We just took the spec and coded it. Coding was easy. But the failure of BDUF was that it still didn't really match what the customer wanted.

Then enter agile and YAGNI, in that limit, coding is easy, just write what the user story says. But then you have to refactor from what was left behind on yesterday's user story. So smart engineers would cheat a little with YAGNI and say, yes we will put an abstraction in because the next user story. I would say that "good engineers" or "good coders" found that balance in abstraction to move fast but make abstractions not overkill. And I think that's what all the wailing and gnashing of teeth is right now: the good engineers aren't needed anymore.

I can tell an LLM to code something, and as I add complexity it's happy to refactor and manage the context so we don't need to worry about "clean code" or "quality code". As long as what the LLM writes meets the spec, then we're happy.

Ah, sorry long rant and ramble. But I just think the "hard part" has been managing context, and the context we're managing context is just changing. Those good at managing context will be good at coding with LLMs, and those that weren't won't be.

Rather than "managerial" I tend to view it as more "professorial". Maybe it's my academic training (but I got out before I became a professor), but I tend to like to think about my problems in terms of abstractions and invariants and common ways to represent things. I just now describe them in English rather than code. But I do find that LLMs do a much better job the more clearly and completely I express what's in my head. I get that same satisfaction that you mention, but now it's when the LLM does what I want on the first try rather than fighting with it all day.

In the past when my junior engineers were fighting with a bug all week, I always told them the dopamine hit of solving the bug is what keeps you coming back to work for 20 years. Half joking, but the dopamine hit of from the satisfaction getting complex working software in a weekend is like crack to me, we'll see how that plays out long term :)

I think the trouble is living wage means different things to different people.

I look back to my time in grad school. My wife and I were lucky that we were able to get fellowships so we were able to focus directly on school and didn't have to do another job or teach. But it was _tight_! I remember keeping track of every cent in a spreadsheet and we generally were at around $100 extra to save every month (which would then be eaten up by a random car repair or something). We never went into debt, but we really paid attention to our spending. No fancy going out, no travel, no avocado toast.

I think right now people put too much "fun" in the definition of living wage. So yes, I wouldn't want someone to be worried about making rent and eating while being apprentice, but it's also not going to be a glamorous instagram lifestyle.

In 2000-2005 dollars we were making a combined $30,000 a year as grad students. I don't know what that means now, but that's probably the actual value we provided to our advisors (paper writing, conference posters, etc).

My (probably unpopular here) opinion is just the opposite. We need more of an apprenticeship model where you're not paid a bunch because you're still learning, and you probably bring negative-to-zero value initially. When a fresh-out-of-college junior engineer brings in SV style money, the expectation should be that they already know what they're doing.

In the trades you start off low pay because you're generally more in the way than helpful, but then you gain the experience and knowledge to be valuable.

Even the resident-doctor relationship is like this. Resident are overworked and poorly paid because they are more distracting than the value they add, then there's the big reward at the end.

The grad-student/professor model is kind of like this too except for all the pyramid scheme stuff that happens there.

I think most technical fields need to go to this model where the newbie commits to learning and trying to be valuable instead of rest-and-vest. And then once they're valuable they get paid more in proportion to the value they bring to the field.

In my small company we had to switch about 5 years ago to only hiring folks with lots of experience (10-15 years). We tried hiring younger fresh-out-of-college engineers, but "market rate" was too high and they required too much attention from senior staff and it made us unsustainably unproductive. We wanted to mentor and teach the next generation, but we couldn't afford it.

What does the N in NP stand for?

I'm a bit joking, but we've been working in deterministic computation for so long, we don't even think of there being another way.

But seriously, I do view AI as the input to a deterministic machine. Junior engineers (well all engineers) aren't deterministic, and we've made processes to direct their behavior towards making better software. AI agents do a better job of following my processes than engineers. We move up the stack towards testing and verification rather than writing. That doesn't make me sad, after 40 years of coding, I'm kind of tired of it. I have more ideas than I can code, so I'm happy to give AI my ideas and have it code for me.

I had a former manager tell me that all technology problems are really people problems, now maybe all technology problems are all agent problems and we just have to get comfortable with managing agents like we got comfortable with managing people.

In my experience, I'm using LLMs as my abstraction to "junior engineer". A junior engineer isn't deterministic either. I find that if you treat the LLM output like a person's output, you're good. Or at least in my projects, it's been very successful. I don't have it generate more code than I can review, or if I give it a snippet to help me fix it, if it ends up re-writing it like an ambitious engineer would do, I tell it to start over and make minimal changes.

I guess I'm not spun up about the determinism because I've been working at the "treat it like a person" level more than the "treat it like a compiler" level.

To me, it's really like an engineer who knows the docs and had a good memory rather than infallable code generator.

I work at a small company, so we don't have tons of processes in place, but I imagine that if you already had huge "standards" docs that engineers need to follow, then giving the LLM those standards would make things even better.

Recently I did something similar to this on a small part of a project that has built up organically over a couple of years. One developer was getting the output of one program (that we didn't control) into one of our systems. So he wrote a python parser to read the output and put it in a structured format for us to read. Over the years edge cases and options kept creeping in, small features were added one at a time and it kept working.

We had to migrate to a new customer who didn't want the python middle layer, they wanted our software (Java) to read the output directly (don't argue on why this was necessary, just a requirement from a big customer). Since the python script grew up over time by many developers with small changes for new features, it was nearly impossible to understand exactly what it was doing.

I took only the parser code gave it to Opus 4.7 and said "This python code parses an undocumented file format. All of our tests pass and this code is the spec for the file format. Read the python code and create me a BNF document that describes what it parses." I took the BNF and started a new session and said "write me a parser in Java that parses the grammar in this BNF." It got it in one shot. Then all I needed to do is take what was parsed and pipe it into our existing software that took the structured format the python created.

Most of what I do is brownfield work and I really can't put our whole code base into an LLM because of IP issues, but for little things like this, is saves me weeks of work.

Is more code really bad? For humans, yes we want thing abstracted, but sometimes it may make more sense to actually repeat yourself. If a machine is writing and maintaining the code, do we need that extra layer now?

In the olden days we used Duff's devices and manually unrolled loops with duplicated code that we wrote ourselves.

Now, the compiler is "smart" enough to understand your intent and actually generates repeated assembly code that is duplicated. You don't care that it's duplicated because the compiler is doing it for you.

I've had some projects recently where I was using an LLM where I needed a few snippets of non-trivial computational geometry. In the old days, I'd have to go search for a library and get permission from compliance to import the library and then I'd have to convert my domain representations of stuff into the formats that library needed. All of that would have been cheaper than me writing the code myself, but it was non-trivial.

Now the LLM can write for me only the stuff I need (no extra big library to import) and it will use the data in the format I stored it in (no needing to translate data structures). The canon says the "right" way to do it would be to have a geometry library to prevent repeated code, but here I have a self contained function that "just works".

How much of "good engineering practices" exist because we're trying to make it easy for humans to work with the code?

Pick your favorite GoF design pattern. Is that they best way to do it for the computer or the best way to do it for the developer?

I'm just making this up now, maybe it's not the greatest example; but, let's consider the "visitor" pattern.

There's some framework that does a big loop and calls the visit() function on an object. If you want to add a new type, you inherit from that interface, put visit() on your function and all is well. From a "good" engineering practice, this makes sense to a developer, you don't have to touch much code and your stuff lives in it's own little area. That all feels right to us as developers because we don't have a big context window.

But what if your code was all generated code, and if you want to add a new type to do something that would have been done in visit(). You tell the LLM "add this new functionality to the loop for this type of object". Maybe it does a case statement and puts the stuff right in the loop. That "feels" bad if there's a human in the loop, but does it matter to the computer?

Yes, we're early LLMs aren't deterministic, and verification may be hard now. But that may change.

In the context of a higher-level language, y=x/3 and y=x/4 look the same, but I bet the generated assembly does a shift on the latter and a multiply-by-a-constant on the former. While the "developer interface", the source code, looks similar (like writing to a visitor pattern), the generated assembly will look different. Do we care?