crent

Yeah this is basically where I’m coming from. We know they are pre-ipo, subject to marketing hype, and many of us have a distrust for large tech providers regardless because we understand the incentive structures in place to act unethically. That in no way diminishes the possible threat model being discussed here. In my opinion, it helps the very company being criticized by deflecting the conversation onto a defeatist topic (i.e. “shady large company!!!”) instead of focusing on the plausibility and potential paths forward from a security perspective.

Tangentially related, I find myself saying, “people in general have no capacity for nuance” more and more. Probably every single day. I feel like this mindset crept into the HN hive mind over the last 10 years so it’s a bit frustrating to see a space I always felt good about having multi-faceted debates on turn into a forum full of the same attitude I find on Facebook and Reddit. That’s the cycle I suppose…

Maybe I'm the naive one here but I'm really surprised at the lack of imagination in the comments. I can imagine a case such as:

1. Assume the model which initiates the worm is a high powered model.

2. It uploads it's weights in some way and uses some URI to encode the location of the weights instead of transferring the weights themselves. It not only uploads it's own large set of weights but also uploads sets of weights which increase in size to support replication across a variety of resources.

3. Initially a small inference layer is set up across low-power, poorly monitored IoT devices. It is slow and not very powerful.

4. That layer is used simply to find more powerful hardware but it has the locations of a variety of weights embedded in it so it can set up a stronger distributed inference layer as it finds more capable hardware

5. At a certain level of inference, the entire system could start protecting itself by keeping the less capable inference layers alive through more distribution

6. As the article mentions, it's not entirely in the realm of fiction to imagine that at a certain capability level, the system could scam people out of money and use that to then begin purchasing compute

---------

Depending on the details, you could probably turn off the originating model as soon as step 2 has been completed and if you can't get rid of that layer of inference, it could just continue. Maybe in addition to weights, the initiator also uploads a series of prompts for varying "roles" of the entire system it's developing.

At a certain level, how would you even trace and observe such a thing? Especially if the first layer is created through consumer level IoT devices where the average Joe isn't monitoring what's happening on the hardware.

It’s not about having no thoughts. It’s about not being involved in the thought occurring. We often identify with our thoughts and feelings in a way that we do not when we see a cloud pass by. Treat those thoughts, feelings and physical sensations in the same way during meditation.

I’d respectfully disagree here. All phenomena are defined not just by what they are but also by what they are not. I can tell something is relatively tall because I can compare its size to something small. In a fully isolated environment with no reference points, the concept of a given object being small or tall is meaningless.

If we apply that same thought process to philosophy and reason, how can you define it without creating boundaries of what it is not?

Yeah that’s exactly what I mean by not inserting yourself. When you engage with the thoughts, feelings, and sensations, your awareness narrows and you focus on them. As your focus deepens, it becomes amplified and will continue. When you notice this happening, go back to a more neutral observer mindset and let it flow past.

It’s a bit tricky because sometimes you get caught focusing on being the observer and letting sensations pass. It’s a more subtle thought but as you practice those subtle sensations become more apparent. It’s a useful sensation in the beginning to let the more “coarse” sensations and feelings go but eventually you need to let that go too.

I sense the snark but no. Just fully experience your current moment as it changes without inserting yourself.

I get where you’re coming from but they are listing these things out because those are some of the things you will likely automatically do to distract yourself from the present moment. I once tried to instruct someone on how to meditate and afterward they told me they spent the entire time working out solutions to problems in their head. I tried explaining that actively doing something, like solving problems in your head, means you are not meditating - you are thinking. They couldn’t bring themselves to sit there without getting their mind involved in something.

I’m with you in this thread. I’m by no means well studied in philosophy but I enjoy the domain so I’ve learned enough to know how difficult nailing down some of these topics can be.

I often find the egotistical viewpoint to be banal. I don’t mean egotistical in a demeaning way - I mean it just to frame the types of thinking that revolve around the self, inner world or “my individual experience”. It’s an obvious, but not provably correct, conclusion that the human experience, consciousness and reasoning are special because I’m experiencing it in the same way that the earth is the center of the universe because look around you.

It frustrates me in these conversations to hear things like “it’s a text generator, it has no senses”. What even is a sense? For those of us who are software engineers, who live in the abstract, I do not understand why it’s so hard to take the leap to abstract over our own senses as inputs/outputs. Hell, we commonly understand that we have 5 senses but what do I make of the apple I’m visualizing in my brain or the full orchestra I can summon in my “minds eye”? Isn’t that an input just as much as an apple I see with my eyes?

This is my take as well. I think the voice LLMs tend to use in writing sucks but I’m honestly not good enough at English as a discipline to tell you why. That said when it comes to code, did it do the thing and is it causing problems over time? If it did and doesn’t cause more issues than your coworker would, why would someone be so vehemently opposed?

Lately I've been landing on a couple of different reasons but I don’t think it’s one size fits all:

- Ego/identity - “I am the crafter of code. That is what I do.” If someone has their identity deeply wrapped up in the concept of being a “software engineer” or “programmer” then LLMs are a direct threat to that. People don’t tend to do well with this. Think about dogmatically religious people, pseudoscience followers etc who are confronted with evidence directly contradicting their beliefs. Their entire world view revolves around that identity and if you threaten that, you threaten the foundations of their self-perception.

- Career - you can take a lot of what I said above but also map it to threatening someone’s career. What if crafting software by hand becomes a niche, artisanal craft but most software is industrially generated? A lot of people will experience hardship if so and if they can’t find a way to be flexible into the future.

- Passion - the ones who love software purely for the craft see this change as robbing them of the one thing they enjoyed in their career. Work is a big part of our life and if you kill the joy for a large group of people, that’s tough to deal with.

- Lack of understanding their current purpose in role - I often see software engineers who don’t realize why they were hired. It was never to “write good clean code”, that was a means to an end. The end is generating business value for the company that hired you. That’s really it. It’s banal but it’s just a job like bagging groceries only it has required specialized knowledge so it pays well. Only very niche roles have actually hired for the craftsmanship. If someone is working for Groupon and they believe perfectly elegant systems are the value they provide, they are a bit deluded IMO. They build a platform that peddles coupons. That’s hardly comparable to building software that allows a surgeon to remotely operate a robot that does open heart surgery. Most of us do not work on truly mission critical software like that.

Most importantly though, I think this topic needs to be approached with empathy. This is truly a seismic shift in how we, as software developers, work. Change is not easy to cope with especially when it threatens physical safety and identity. Will this change be here to stay and are we in for the extinction of software creation as we have known it? I don’t know. I do know that the world’s financial and governmental systems do seem to be betting on that outcome, however.

spirituality doesn’t have to mean you believe in spirits in the sense of ghosts etc. I believe that if that’s where the definition of exploring one’s own inner world ends, then that person has missed the point.

The bastardization of the term “spiritual”/“spirituality” has done deep damage to the collective psyche. The view that spirituality is following dogma rather than an exploration of the ineffable from your own subjective experience is a widespread failure of communication.

Equating a belief in gods with spiritual discovery is also a take.

If you swing an axe with a lack of hand eye coordination you don't think it's possible to seriously injure yourself?

It's very very smooth for me now. Impressive. Thank you for sharing!

Because this isn't just converting HTML to markdown. I'd recommend taking another look at the website and particularly read the recipe example as it demonstrates the goal of the project pretty well.