nadam

Does anyone know why these Claudisms exist? Are these kind of expressions a result of some kind of 'evolution', so they are more effective in chain-of-thought reasoning that other expressions? Or the preference of people doing some kind of human-feedback in the post training? Clearly probably not the 'average' expressions of their training data, and clearly if there would be no upside and it is easy to remove them they should be removed, as they are annoying, so I guess there is an upside or hard to remove them...

You put the "agent" word into apostrophes as if I use it as a marketing buzzword. No. An agent is an LLM in a loop with memory usage, with file system access, etc, which is usually more effective than just using an LLM as is, especially if you orchestrate these agents and subagents in a good way. In my opinion using the word 'ChatGPT' (a specific user-facing LLM brand) in these discussions is much more buzzwordy than using the word agent.

Maybe. But I am working with these agents and I see the sophisticated patterns using LLMs, using ground truth data and human experts working together. It could be much more effective than 'patient asking chatgpt' vs. 'general doctor one-shotting a problem without AI assistance and enough context'.

I am amazed at the false dichotomy (ChatGPT vs. Doctors) discussed in the comments. Let's look at it from the perspective of how a complex software is developed with a team of software engineers and with AI assistance. There is GitHub, Slack discussions, Pull Request, reviews, agents, etc...

Why a patient's health cannot be seen this way?

Version controlled source code -> medical records, health data

Slack channels -> discussion forum dedicated to one patient's health: among human doctors (specialists), AI agents and the patient.

In my opinion we are in the stone age compared to the above.

I find it amusing how people take this 'leveling game' at big companies so seriously. The title they get at these companies become their identity. They live by the rules the company imposes on them. Amusingly almost the opposite of independent thinking, which they are so proud of. What I found during my career is that some people blossom at these big companies, and some, with equal talent cannot really realize themselves, while they blossom at other organizations, perhaps in startups, or just at more interesting problems. What some people do not realize is that this aspect that as you level at these companies 'you get the big picture more and more' is not really true in a lot of cases. Efficiency and big picture thinking is almost orthogonal. Efficiency is context dependent, high level thinking is less so. I have seen people getting the big picture on things even at low levels or even considered a junior, while people on high levels suprisingly small minded. Higher level at these organizations means that you are more productive in the given field, given organization, for some reason, this can be because of good political skills, because you work your ass off, or because you have a very efficient brain, but it does not necessarily mean that you have better high level vision, or taste or maturity even. Sometimes it is almost laughable how these people who treat these leveling system seriously and get to a relatively high level treat young colleagues. They are almost naive about how young people think. They think that L3 juniors cannot do anything alone. While I see even young kids playing with suprising autonomy if they are thinking in the right problem space.

True role models of mine never were/are L3, L4, L5, etc... They were/are awesome engineers or scientist from the get go. (Like the Johns: The von NEumann and the Carmack.) Experience lets you get better and better, but if you are stuck at a role or organization, the problem might be that you need to find something that you are more passionate about and not necessarily that you are low level because your thinking is not 'independent enough' or you are not 'high-level enough'. At least that is my experience.

I was just thinking about what AI assisted coding brings to the design discussion, especially as AI become more and more powerful and we rely on it more and more. You still want to make things modular and easy to understand so that AI understands it easily and the needed info to modify a module can fit in a relatively small context window, but the difference is that it is very easy to make large scale measurement about which code style is understood better by an LLM, so maybe some of these debates will be decided relatively objectively!

For the topic: the discussed topics are relatively trivial surface level stuff, mostly I agree with POSD, but these will be handled by AI anyway. I guess humans will use the spare brain capacity to deal with the real deep design questions (for a while).

What I am starting to internalize nowadays is that working faster is useful, but what can really change one's life is learning faster.

Hmm, I think 'fail fast' and 'embrace the grind' are popular and somewhat contradictory advices. Which is better? I think 'fail fast' is (or was?) a bit overhyped so I tend to err on the side of 'embrace the grind'. But obviously the art is in deciding which one to follow in a case by case basis. Working on your dream game for years only to find absolutely no traction is not a good place to be in, but constantly chasing low-effort ideas without any 'moat' can be also fruitless. Moat usually comes with time, effort, and resources invested.

"But when you have to scale the work, then the code has to be easy to read, easy to extend, easy to test, have extensive test coverage, have proper dependency injection / be easy to mock the 3rd party dependencies, be easy to configure so it can be deployed in every cloud provider (i.e. by using env vars and config files for modifying its behavior).."

Interestingly I do not find that the stuff you mentioned are the things that LLLMs are bad at. It can generate easy to read code. It can document its code extensively. It can write tests. It can use dependency injection especially if you ask it to. What I noticed where I am currently am much better than an LLM is that I can still have a very nuanced very complicated problem space in my head and create a solution based on that. The LLM currently cannot solve a problem which is so nuanced and complicated that even I cannot fully describe and work partially from insincts. It is the 'engineering taste' or 'engineering instincts' and our ability to absorb complicated nuanced design problems in our head that separates experienced developers from LLMs.

Unless LLMs get significantly better and just replace humans in every task, I predict that LLMs effect on industry will be that much less developers will be needed but those who will be needed will be relatively 'well paid' as it will be harder to become a professional software developer. (more experience and engineering talent will be needed to work professionally).

"When every supermarket aisle looks like a sea of sameness, when every category abides by the same conventions, when every industry has converged on its own singular style, bold brands and courageous companies have the chance to chart a different course. To be different, distinctive and disruptive."

I don't think it is that simple. There is stuff with different asthetics today, they are just not always successful among the mainstream audience. It is always a choice to substantially differ from the mainstream look, and in industries where the cost of entry is not too high plenty of people do it, they are just not successful or too niche most of the time. Where there is a high cost of entry obviously less players try to be massively different, because being too niche is not profitable in that case.

An example is the material editor. When dragging a value of a property of a node (like color) you see it immediately in the editor. You apply a material on an object, (regular object, water, Landscape), and while changing stuff in the node graph, you can see it applied to the object on the scene almost immediately (depending on some factors of course). A Material can be as complex as a Landscape auto-material which magically auto-generates the surface appearance of a complex landscape (with even foliage). But this is just an example, there are other node graphs in Unreal, like PCG. Hot-reloading my C++ code is not strictly necessary as I write C++ code mostly for really messy complex stuff, where I need to think a lot between tests. So I think Unreal mostly has immediate feedback where it is most necessary.

Re-watching Bret Victor's "Inventing on Principle" was an interesting experience today. At the time of recording (2012) his talk was magical and ahead of its time. Today his principle (immediate feedback to the creator) is fully applied in tools for technical artists. (Unreal Engine, Houdini, etc...) It is funny to see what happened with his ideas: basically what happened is that the software engineer and technical artist roles has got really separated. For technical artists his ideas are everyday life today, almost trivial seen from 2024 (although they are not), although they work mostly not in text-based, but node based programming languages. For software engineers his ideas are not so relevant as software engineering is specializing on the more messy, more abstract or more complicated problems. Frontend development is somewhere in between. I think Tim Sweeney (and his team) is missing from the list, who I believe has seen this coming already in 2012 and built Unreal Engine to its current form.

"compare this with enterprise software, which is orders of magnitude more complex than video games in terms of business logic" Maybe this was true 20 years ago, but I do not think this is true today. Game code of some games is almost as complex as enterprise software or even more complex in some cases (think of grand strategy games like Civilization or Paradox games). The difference is that it still needs to be performant, so the evolutionary force just kills programmers and companies creating unperformant abstractions. In my opinion game programming is just harder than enterprise programming if we speak about complex games. (I have done both). The only thing which is easier in game programming is that it is a bit easier to see clearly in terms of 'business requirements', and also it is more meritocratic (you can start a game company anywhere on the globe, no need to be at business centers.) And of course game programming is more fun, so programmers do the harder job even for less money.

For people who think game programming is less complex than enterprise software, I suggest the CharacterMovementComponent class in unreal engine which is the logic of movement of characters (people) in a networked game environment... With multiple thousand lines of code in just the header is not uncommon in unreal. And this is not complex because of optimization mostly. This is very complex and messy logic. Of course we can argue that networking and physics could be done in a simple naive way, which would be unacceptable in terms of latency and throughput, so all in all complexity is because of optimization after all. But it is not the 'fun' elegant kind of optimization, it is close to messy enterprise software in some sense in my opinion.

His system prompt is too specific. It worked for me with GPT 4o using this prompt:

----

System prompt: Please note that words coming to you are tokenized, so when you get a task that has anything to do with the exact letters in words, you should solve those using the python interpreter.

Prompt: How many r's are in the word strawberry?

----

This whole thing is a non-problem. Adding in this hint into the system prompt the whole topic is solved once and for all. I would argue that if the training data would include this wisdom, it would be even a non-problem without the system prompt.