hathawsh

I've found that AI-written text consistently fails to comprehend the context. It either assumes there's little context available, so it writes too much, or it assumes the reader has all the context, so it writes too little. Human-written text has those problems far less often.

I hope the LLM builders are paying attention and they add some kind of step in the process that causes the system to reflect on its assumptions about context and work to refine its understanding of the context before generating output.

Or it could indicate that more resources have been put into cryptography than most other math fields, so AI hasn't yet caught up to our progress.

Well, to a CMS developer, a lot of problems look like CMS problems. I know because I used to be paid to think like that. Portfolio website? CMS. Support ticket database? CMS. Newspaper layout system, intranet portal, blog, restaurant menu, identity verification system, etc? Yeah, we built all of it using a CMS. It made a lot of sense at the time because HTML and CSS were young and difficult to wrangle. Today, most of the reasons to build or use a CMS have evaporated.

That's a great perspective. Come to think of it, I wonder about the quality of the build123d code it generated when I vibe-coded a prototype complex assembly. Now I'm curious.

Yes, but that's in psychology. Different fields have their own terms without causing too many issues. Communication breaks down when we reuse terms for different things within the same field.

The LLMs have been trained on all kinds of code. They often choose a hacky solution because that's what they encountered in similar situations in their training. It generally only takes a few words to guide them to a better design than what they saw in the wild, but if you don't guide them, they follow the herd.

I will be unhappy if they steal the meaning of NLP. NLP means Natural Language Processing and LLMs are only a subset of NLP. I would have concerns about anyone who claims to have technical knowledge and asserts NLP means "natural language programming." Those are the same kind of people who might suggest Lisp should add mutable objects.

There is a large market for fine art. For everything else, there are printers and copiers. It's very difficult to maintain a career of painting and selling works for $100k. It's much easier to maintain a different career that happens to include printing things with little effort using an ordinary office printer.

That's where software development is going: of course you can still code by hand, but it will be increasingly difficult to get paid to do it. People will have a somewhat different career that happens to include coding things with little effort using an ordinary coding agent. Companies won't be able to afford to pay for expensive coders anymore (because otherwise their competition, who is getting along fine with a small software development budget, will take all their customers.)

I thought the same until I had kids. I realized that kids generally want to learn, but they usually think what the teacher is telling them is not the best thing to be learning. No technology, not even AI, overcomes the student's ego; in fact, technology rarely even tries to address that issue. I think this often applies to adults too. Teachers continue to be necessary to help students achieve an appropriate level of humility to be able to learn.

You clearly have enough humility in your field to be able to soak up knowledge. That's great for you! I suggest your dedication is rare and teachers continue to be essential for society.

While I agree that a coding model (such as Opus) by itself tends to act very shallowly, when it's driven by a harness like Claude Code, the combination seems to be a far more general thing than a LLM. It's capable of consistently making excellent data structure and architectural choices over large code bases. It imitates thinking about anything and it can drive itself for hours.

Honestly, if I simply fed it a sense of presence (I would repeatedly tell it what's going on right now and ask it to react if it thinks it should), it would feel eerily like AGI.

I try to remember to say something like this: "This is an interesting solution, but I'm not confident we have a shared understanding of the problem we're solving. I have a few guesses, but I don't want to bias the discussion, so please restate the problem without suggesting a solution. If it's too difficult to explain, I can suggest what problems I think you might be running into." I wish I always remembered to say that. :-)

Thanks for the correction. In my own words, to commit to memory: once "the cat is out of the bag", the point is not that it's difficult to put the cat back in, the point is that we now know it's a cat and putting it back is not relevant.

"The cat is out of the bag" doesn't mean the cat is gone, it means the cat has freedom. I read the GP as meaning "no single company can currently control the trajectory of AI." That's not unfortunate -- that's great!