bondarchuk

I brought up Unit 731 because an argument like "there's so many potential insights to be gleaned from research like this" is also applicable to it, yet it is obviously deemed invalid by moral standards nonetheless. It is not a 1 on 1 comparison but a more subtle argument, I thought hackernews would be able to handle it...

Sure (it was a mistake to say "sufficiently complex" without qualification, obviously only certain very specific complex assemblages will actually be conscious). But if you think the "honed by billions of years of evolution" thing is relevant (on which I agree with you), you also have to acknowledge that the same goes for the neurons themselves. And not just the neurons in isolation, but also their capacity for self-organizing into computationally useful structures, possibly in response to certain stimuli, which you can be sure this type of research will try to take advantage of.

I'm sure they know more about brains than I do but I'm also sure they cannot derive from an abstract description of a computational process whether that process implements suffering or self-awareness or whatever, which is what I think we'd need to do this human-neurons-as-lego-bricks type research safely.

Or if it ran a couple million soft-body simulations with the goal of creating a reusable temporary link between certain kinds of materials, I think it would probably get there. Just another argument from lack of imagination on the part of the author.

Is it really so wacky to suggest we might want to be careful with bootstrapping complex computation from human neurons? It's quite different from an LLM and the fact that it's not made out of neurons seems to be one of the core arguments for opponents of the possibility of LLM consciousness...

>that just pops out if you have a large enough system.

certainly not a claim I would make.

>No proof is required

Isn't it legitimate to reason on a more abstract level and try to make inferences that way?

>psuedoscientific tests

no clue what you're referring to here or in the rest of your comments about saints and authorities and sins. Another parallel to religion could be drawn w.r.t. the idea that only human souls could ever feel suffering but I don't think it's such a productive level to have this conversation on.

Certainly it wouldn't be the end of science, we could still do all the other science that doesn't involve trying to massage human neurons into doing complex computations.

You could do that if you knew enough about how it works in detail. In practice they're just gonna make as big a blob as they can muster and apply RL on it or something, and nobody will be any the wiser as to what the thing's feeling.

Sure, and you could say the same thing about unit 731. The argument is not that it is inconceivable that there are benefits, but that they should be weighed against the drawbacks, which you don't mention at all.

One of those research areas where nothing they currently do is unethical, but also nothing they currently do is useful, and right about the point where it might ever become useful is also the point where it becomes hugely unethical. So we'd do well to outlaw it right now, but seems like we don't really have a moral framework for cases like this.

There are ways of approaching this attack you're not currently considering. It simply won't work to go "here's how I would do this, that's impossible though, therefore this whole range of attacks is impossible".

So you both thought about it for about 10 seconds and came up with some surface-level objections. Doesn't really prove anything when dedicated attackers with 10s of millions in potential profits are in play. Probably you would've written very similar dismissals about the idea of implanting a mini esim modem in a hardware wallet if someone would've brought it up back then.

Would be good if Apple (or anyone) could have some kind of user-verifiable supply chain integrity thing, if they don't already.

Or just a single apple store sales employee...