AgentMatt

If law was code, there would be no need for judges to make decisions. Instead, a "judge" would be someone making a mechanical derivation of the judgement given the situation and the law.

The law is rather a complex set of guidelines which need to leave some space for interpretation, so that additional relevant information for the situation at hand can affect the decision.

How do you know that it's the words driving the thinking, rather than the stream of words just being an observable trace tacked onto the actual thinking?

Ridiculous? Wow. I'm paying ~$0.4/kWh in western Europe...

> But all inconsistent understanding is incorrect.

Why do you believe that?

Even if we assume that reality itself is self-consistent (what does that even mean?), why would that imply that we humans are able to find a self-consistent representation of it? Maybe reality is self-consistent in some sense but cannot even be represented by the tools we use for theory building.

My point being, the ultimate target of our understanding may be self-consistent, but the way we _necessarily_ have to reduce it to lossy theories means that we can only ever approach it with a non-zero error. And a theory focusing on one aspect, minimizing representational error from one direction of approaching it, necessarily has to make assumptions that will contradict those made by another theory trying to minimize representational error coming from another direction / domain.

> I've observed many (most?) people seemingly being completely oblivious to self-consistency issues of their beliefs and mental models, or even texts they're reading or instructions they're following, and yet... somehow they're generally more successful at life because of it

What do you mean by "successful at life" here? Genuine happiness, fulfillment in life? Or in the sense of doing well by what society holds as it's current interpretion of what one should strive for, and otherwise just kinda drifting through life?

Because if it's the latter, I'd say that is to be expected. It's much simpler to put your energy into fulfilling the expectations of whoever is your superior in your current group, mostly get the expected reward, and then just coast. Reflection and experimentation, which is required to get to self-consistent views, takes effort and and the willingness to question existing beliefs, which will also be uncomfortable times.

Which pictures are you missing? I see three stills from the Bladerunner intro sequence, and two embedded videos of it.

What makes it so token hungry? Is she directly using the LLM for genome analysis rather then having it write the data analysis algos?

In your experience, would you say that your communication has become more precise?

Reflecting on it now, I believe that metaphors and idioms can help a lot for communicating intuitively and they can convey a lot of information. But at the same time in the situations where a idiom isn't understood by the other, I often have to take a moment to extract the essence of its meaning.

That's a very good question that I also don't have an answer to (yet). But I believe part of the reason for that is that were currently still in the middle of this shift, so there's issues of both knowledge about AI and how people relate to it. And of course there's so many different situations here.

Your non-technical parents ask you a question? Use the best tool for the job (might be Claude), then just help them with the task.

Talking to a friend looking for genuine advice? Maybe show them how you use Claude while also answering the actual question.

A lazy person trying to make you do their work? Don't become their Claude interface.

Someone asking a question but actually just trying to open up a conversation? Maybe check-in what their intention behind the question is, and if it's indeed conversation / exploration, leave the phone in your pocket and do that.

They are most likely not based in the US, but converting to USD to make comparison easier.

Seems like for such requests it's necessary to get some proof of work: require a meeting where for every artifact they sent you to review, they briefly explain the gist and point out the motivation for creating the artifact.

Agreed. It seems a core issue underlying these prompt injection attacks is a failure to properly scope the agent's permissions. In this case, depending on what exactly the agent is supposed to actually do, this might be defining a separate workflow agent per repo, or a workflow agent with broader repo access but configured to only be triggered by users on an allow list (still compatible with developing in the open, still allows outsiders to open public issues, but takes into account the different trust to be placed in each). And likely many more options when one properly thinks about it.

But that requires:

1. the technical ability for such fine-grained scoping / permissions

2. actually taking the time to think about what you want to achieve with the agent and what the smallest set of permissions / capabilities is for it to achieve it

Regarding 1., I think this will come, we're still in the wild west phase of agent usage. It'll be interesting to see which abstraction(s) will turn out to be the best interface for humans designing agents (minimize friction for finding and defining scope and permissions) and to limit agent capabilities (again finding the best trade-off between level of detail possible for defining capabilities and the ease of use of actually doing it).

Regarding 2., well, that's still the core problem that's always prevented the construction of high quality software, isn't it? Taking the time to properly think it out,and then taking the time to properly implement it. Which goes counter to the "move fast and break things" approach of people throwing agents at everything.

I agree with the basic premise: that when using LLMs, they will tend towards some mean when resolving ambiguity.

There's a very interesting opportunity here for a deeper investigation.

- What does "regression to the mean" actually mean in practice when the LLM is conditioned on a possibly large amount of context?

- How does this perceived regression to the mean affect the result in different applications? When implementing code, it may show up as keeping it simple, hence easily understandable, "nonclever". When writing documentation, it may show up as simple language, short sentences, etc. supporting the intent of communicating with little friction to a broad audience. When brainstorming product ideas, it may show up as regurgitating old and boring ideas, but dressed in fancy language and affirmations that hide the shallowness of the content.

- What can be done to alter this behavior? Now that temperature doesn't seem to be a parameter anymore in new models, how can we steer creativity of the model?

- If the model's creativity is fundamentally limited, is there a way we can use it to support us in the expression of our creativity, leveraging the different strengths of humans and LLMs in a way that the result transcends the limits of either?

Unfortunately, I don't see the article doing that. And, while I know pointing out LLM-isms is often a cheap shot these days, I feel compelled to point out that this article is full of what I perceived as LLM-ism, quite ironic given the premise and the statement ("written off-distribution · on purpose").

E.g.

> Trained on the past, it answers in the past tense of thought. Not what is true. What is typical.

> We converge — not on what is right, but on what is average.

> Not the answer it was sure of — the one it would not stop correcting

I'm sure there's plenty of writing in the above style to be found on the Internet, and hence having been trained on by the LLM. I'm also not a fan of this style, and in particular I'd say it's rarely or never found in scientific / technical writing meant to convey understanding rather than sell or hype. So here it's IMO more of a style mismatch.