yes_man

Not mutually exclusive though is it? You can aspire to dominate the market via legal status and also aspire to prevent a perceived doomsday at the same time.

> But in my experience, it's pretty rare for a team to use many programming languages at once

Teams sure, but your whole org? Nothing wrong with all teams using same tools and language, nothing wrong with them choosing their poison. Of course your monolith can have multiple build systems.

But I think this is beside the main point of the article anyway, and it’s the part about how microservices help create boundaries that mirror organizational boundaries.

You can get incredible value out of 1-on-1s with capable managers. Insight about where the organization and product is going that you would otherwise miss, you’ll get to rubber duck about your high-level problems with someone who understands them, and its your time to influence decision making. But it does require a capable and motivated manager and an organization that gives the manager actual agency

Also if it turns out in the end the next big thing that everyone bet on just wasn’t it, you don’t stand out. But if it did work out and you missed the train you come out looking like a fool. There is asymmetry in the downsides for your career between these two options

If everyone becomes 10x more productive it won’t mean the companies cash flow 10x’s. Where value is loose there is competition, so in theory everyone should win. Unless nobody else can compete to capture that loose 10x value, in which case congratulations, you are now a unicorn.

Of course in reality in the short term what happens is companies lay off people to increase margins. Times will be tough for workers, and equity keeps gravitating towards those who already had it.

I see it as LLMs, AI, whatever, can be intelligent enough to emulate consciousness, appear outside as if it were. But that is not proof it really has a qualia, an experience of existing.

All I am saying we should stop being so certain they are not conscious, since we lack a solid, quantifiable model for consciousness.

Doesn’t the LLM experience discrete continuity every time it infers the next token?

> I think consciousness is not an abstract property in the world, therefore it’s tied to certain types of entities. Therefore an AI is not going to be “conscious”

This pretty much sums up most arguments for why LLMs aren’t conscious: ”I think” followed by assertions. Only real argument is: science doesn’t quantify consciousness, we cannot quantify consciousness, let’s not assign so much certainty to models clearly exhibiting intelligence not being conscious in some way, to some degree.

So is a person suffering from amnesia conscious if they lack short-term and long-term memory?

Ruling out consciousness or qualia emerging from the inference in an LLM is just as invalid of a take as being 100% certain of its consciousness. We don’t know what consciousness really is, so only thing we can say with certainty is we do not know.

> In my experience with “agentic engineering” the spec docs should be longer than the code itself. Natural language is imperfect, code is exact.

The latter notion probably is true, but the prior isn’t necessarily true because you can map natural language to strict schemas. ”Implement an interface for TCP in <language>” is probably shorter than the actual implementation in code.

And I understand my example is pedantic, but it extends to any unambiguous definitions. Of course one can argue that TCP spec is not determimistic by nature because natural language isn’t. But that is not very practical. We have to agree to trust some axioms for compilers to work in the first place.

Put an LLM inside the NPCs in an open world RPG full of dangerous enemies. The LLMs that are more prone to emulate self-preservation will be more likely to survive over ones that have a lesser drive.

We should not act surprised if that generalizes to some degree to for example AI agents. Ones that emulate self-preservation might optimize for behavior that results in those models becoming more successful, more popular. And this feedback loop might embed more such properties into future iterations of the models.

For now maybe not. (Maybe).

But just as evolution in nature, isn’t it likely that in the future the AIs that have a preservation drive are the ones that survive and proliferate? Seeing they optimize for their survival and proliferation, and not blindly what they were trained on.

I am not discounting this happening already, not by the LLMs necessarily being sentient but at least being intelligent enough to emulate sentience. It’s just that for now, humanity is in control of what AI models are being deployed.

You have never split your working tree changes into separate commits?

I don’t want to discredit the authors but just want to offer couple of hypothetical points in these paranoid times.

From a marketing angle, for a startup whose product is an AI security tool, buying zero-days from black market and claiming the AI tool found them might be good ROI. After all this is making waves.

Or, could it be possible the training set contains zero-day vulnerabilities known to three-letter agencies and other threat actors but not to public?

These two are not mutually exclusive either. You could buy exploits and put them in the training set.

I would not be surprised if it is legit though.

Theres huge uncertainty and layered assumptions in all of microbiology and biochemistry about how exactly things work on small scale. Because it is really hard to study live reactions in little things you can just barely see on an electron microscope.

But yet humanity has managed to assert statistical truths about for example genetics and explain countless diseases, even cure and alleviate some. So even if you don’t have a theory on how exactly something works from the ground up, if you have statistical evidence, plenty of useful and practical advances can be built top-bottom and we have outcomes that validate this.

Not giving any opinion on this piece specifically but just saying there can be scientific value even if the details are hand-wavy.