rented_mule

Unfortunately almost any kind of blog. My spouse's blog, which she hasn't updated in several years, came under attack about a month ago. Her blog is just posts of some of our travels mixed in with her arts and crafts. It went from a few hundred hits a day to 20M hits a day coming from 2M unique "users" a day. It wasn't a full scrape, but it was hitting a broad range of pages. The traffic originated from 170 countries, so it would seem to be a botnet. It went on for 2-3 weeks before dropping below 1M hits per day. It still hasn't dropped back down to normal levels.

I think I get where you're coming from. Both the immense respect for Rich Hickey and the disagreement with him. I hopefully don't have to say much for the respect part - he has repeatedly changed how I think about my craft for the better. As I've mentored others, I've often pointed them to talks of his that were relevant to what they were missing. And that's all without using Clojure. But I definitely use ideas from Clojure in other languages where it fits.

On the LLM front, it's been a fascinating journey for me. I started with the private beta of Github Copilot. I was an instant convert for the "better code completion" aspect. But my first attempts with coding agents were rough enough that I stayed away.

I'm now retired, but I want to understand how things are evolving. So two months ago I volunteered to automate a cartography data pipeline for an educator. I have a background in all aspects of the project. It's not simple - there are several data sources from governmental and corporate sources that need to be integrated together, and most of them are updated every few minutes. New maps need to be available within ~5 minutes of a data source updating. All this with many GB of data and running on a low powered machine, taking less than 2GB of RAM.

I bought some DeepSeek tokens and went to work. A few weeks later a mentee went to work at OpenAI and gave me a "friends and family" Pro level plan. I'm converted, and I absolutely see it as a tool in the sense Rich describes. I even question whether any "software engineers" using it, but not as a tool, are doing it right (I'd love to be educated if I'm wrong!).

I'm still learning, but that's part of the disagreement I have with Rich's take. He mentioned feedback. I often watch the "reasoning" process the agent goes through to see the approach it's taking. It's not at all unusual for me to steer it when I see it going in a direction I don't like. I've also found an incredibly powerful pattern where I can explore the approach I want to take for a part of the system with the agent. I have it create prototypes with different strategies (streaming vs. in-memory, different data structures, relying on various libraries, etc.) and then I have it characterize trade-offs between the prototypes (performance, memory usage, conceptual simplicity, code simplicity, etc.). This allows me to make informed decisions about which approaches to pursue further, and my decades of experience in software engineering constantly go to good use here.

It's also given me the time to go far deeper into things I never could have justified. Of course the project has 100% code coverage and branch coverage. It also has hundreds of property-based tests. And thousands of Lean proofs of the behavior of the algorithms, along with conformance tests that the code and the Lean adaptation of that code behave the same way. And lots of TLA+ verification of the overall flow of the system. None of that proves there are no bugs or that this is the right system, that's all on me. A month in I would have said I was fairly happy with the quality. Now I'm proud of the quality.

As I've gone through this process, I've dug into the code for patterns I don't like, and had the agent clean them up, all the while distilling guidance. I've had a lot of luck with that guidance being followed once I figured out the right place to put it and to reference it. I've spent entire days doing nothing but giving prompts to do specific refactoring. The beauty is that large refactorings don't involve the same boredom interrupted by anxiety they used to for me.

English prose (documentation, comments in the code, etc.) is what I've been the least happy with. An hour ago I had the idea to give it ~1,000 lines of my HN comments and ask it to characterize the style and tone of the technical parts. Then I told it to turn that into guidance. Then I told it to rephrase that same guidance in a way that follows the guidance. I would never mistake the result as my own, but it's much, much better than what it was producing before. While I've been writing this comment, it's been applying it to all prose in the project.

This very much feels like a tool that is letting me shape the project (and shape the tool itself). But it's letting me do it at a higher level. It handles the code so I can fully focus on the architecture, algorithms, and quality. This alone makes me think that the best simple guidance for quality results is dozens of prompts per day, not a handful (when a day is 100% focused on development). When you can't think of dozens of prompts, think and explore more (the agent can help with that!). That does require you know what quality is and how that can be achieved.

To reconcile this with what Rich is saying. I suppose he's right that it's not a tool for writing code. But it very much feels like a tool I'm wielding to do software engineering at a far more powerful level than I could have done without it. The resulting project has "me" everywhere in it. The people who know my work the best would look at this project and know that it's mine. Perhaps not the code, but the algorithms, the data flow, the approach to documentation and testing, etc.

For anyone curious, the project is getting close to 200K lines. About 25% of that is the code that is the data pipeline. About 25% is documentation. About 50% is testing and verification. One reason this is possible is that I'm working alone and have full control over the process (what I spend before it became free, how it's built, what my process is, and even the product definition within the constraints of what's valuable to the intended user). All told, it has been about 6 weeks of full time work worth of my time.

The next experiment... a web front-end to this system for the general public. This is not at all in my wheelhouse, so it should be interesting to see how much my blind spots cost me.

For me, vaultwarden's RSS is ~45 MB, with ~13MB of that being shared. I have it running as a secondary thing on a 512 MB machine and don't notice it's there. Is there a reason you presume it's a memory hog?

This is shocking to me. My brother is a retired college football head coach. Two of his sons are acclaimed college football players. I've spent time with the head coaches of two teams currently ranked in the top 10 in the NCAA. I've been around the sport for decades and know how coaches' lives can be monopolized by the job. To see an NFL head coach doing something requiring this level of time commitment blows my mind. I guess it's a reminder that to reach the very top level of a sought after profession, you have to be a remarkable individual.

I had a similar experience having Astra add TLA+ and Lean verification tests to a project with somewhat complex state and many small algorithms (almost all generated by coding agents). The system had been working quite well, but adding formal verification surfaced 33 classes of bugs, some of which had several instances.

I completely agree with you about how fun these things can be. I've long believed in automated testing methods that were beyond what I could justify. I learned about formal methods in college in the 1980s and have never been able to justify (even to myself) applying them. Now a volunteer project I've been working on has 100% standard test coverage, many property-based tests, and 1000s of formal verification tests. That combination has surfaced multiple bugs in widely used underlying libraries and a bug in Rosetta 2's Intel emulation that was affecting me. And that's all just on the testing front. I'm having a lot of fun with all this.

My understanding is that Waymo's remote driving is not direct control of the car, for exactly these reasons among others. So human operators don't have steering wheels or joysticks. Instead the humans can give something closer to advice (e.g., "pull to the right and stop") that the car can accept, modify, or reject.

Going much deeper on tests has been transformative for me. In a solo project started from scratch, I'm 6-7 weeks in, and it's up to ~90K lines. ~60K of those are tests. Those tests have now found (and then the agent has correctly diagnosed) multiple bugs in broadly used libraries that I'm using in my project. That's because those bugs surfaced as occasional issues in my project. Especially powerful are all the property-based tests (perhaps what you are calling fuzzing? I'm using the Python package called Hypothesis for this).

Another spectrum that I've found useful to explore is the scope of what I ask the coding agent to do in one turn. I see some people trying to do one massive prompt that the coding agent works on for a day or more. I find a large boost in overall quality if I do 10-20 prompts per day (not counting the prompts where I'm just trying to understand things). It's still much less of my time than hand-coding, but the resulting architecture looks like my own. The quality of the overall system is great. There are certainly issues here and there in the code, but it's always that way once a project gets large enough. Now it's easier to address any particular issue throughout the code base in one go.

> You do not have to agree with them on everything during the interview.

Absolutely! When I'm the interviewer, I look for a genuine opportunity to amicably express disagreement with the candidate in order to see what it's like to navigate that with them. It's a great sign when we both end up enjoying learning from each other as a result of the exchange.

I think the ideal goal of an interview is to make a prediction of how happy and productive the candidate will be in the role they are being considered for. In a perfect world, that would be a collaborative effort between the company and the candidate, because they both should care deeply about the accuracy of those predictions. Unfortunately interviews often take on an adversarial/guarded dynamic that reduces the quality of those predictions. I believe that LeetCode style interviews make the predictions worse, and not only because they encourage that negative dynamic.

Even if you think that LeetCode evaluates how smart someone is, that's not what companies should be trying to measure. It doesn't matter how smart someone is if those smarts won't be put to good use. One of the smartest people I ever worked with was an amazing fit and launched the team to new heights. Another of the smartest people I ever worked with was a net negative for team productivity because he was such a jerk to people that those people would spend tons of their time and energy to find ways to avoid working with him. Despite all his smarts, he was a terrible match for the role and therefore was not productive. Being a jerk in some environments might even be a positive, but that's part of evaluating fit.

If the mechanics of the proof are not understood, how do we know it's been proven? Math proofs are not a "trust me, bro" kind of thing.

I like to leverage that when brainstorming solutions to hard problems. Instead of contemplating small percentage improvements, try to think about what's in the way of improvements that are orders of magnitude better (e.g., don't take time to run a big task from 100s to 90s, take it to milliseconds). Sometimes it unlocks big ideas.