coffeemug
Joined 19 years ago
Karma
16,039
Submissions
1,469
Member since
Aug 14
spakhm.com, coffeemug [at] gmail [dot] com
View on Hacker NewsRecent submissions
Comment · 1 month ago
The ecosystem to build cheap physical products is incredibly important for creating new wealth. If you're building something new, in manufacturing cities in China you can walk across the street and get everything you need. In the U.S. you have to fly to China and build relationships with Chinese suppliers. Which means creation of new products, which is where most future wealth will come from, is extremely difficult. This is like Germany lacking a meaningful tech ecosystem because creating tech startups is so difficult.
We can make planes and engines because we kept the expertise, and we can make rockets because of herculean efforts of a few people. This simply isn't good enough, we need to make creating new physical things as easy as possible.
Comment · 1 month ago
> completely independent of the question on whether there are any empirical arguments that LLMs are conscious or are not conscious he argues from the end here and says that if they were conscious, this would have horrible consequences for our society, so we must never assume that they are
He is not arguing that. He is arguing that these programs obviously aren't conscious, and that it's dangerous to tilt the weights toward emulating conscious beings for reasons outlined in the essay.
Comment · 3 months ago
A government agency tests all medications, why not models?
Comment · 3 months ago
Imagine a model that's 10000x more intelligent than any human. You integrate that model into a sensitive industry. You now effectively have an extremely powerful foreign agent running e.g. your energy sector. This is qualitatively very different than a backdoor in Redis.
I agree this is science fiction-y, but given the pace of progress I don't think it's so easy to dismiss.
Comment · 3 months ago
I'm not advocating for the policy, I'm just saying that there is a legitimate argument that's not so easy to dismiss.
> And you could still ban specific, known malicious models/labs instead of blanket open-source bans
With publicly available models today, absolutely. With models 10000x more powerful, we would need to be extremely conservative. (In that world everything is upside down, though; I don't know if "banning" is even a meaningful concept in that reality)
Comment · 3 months ago
Subsidizing a product to capture market share is very different from exploiting a national security attack vector.
Comment · 3 months ago
I am not supposing a low-risk event. This is an obvious attack vector that an adversary would almost certainly exploit. I would exploit it if I were them.
Comment · 3 months ago
Suppose a foreign actor deliberately builds an unaligned model, and dumps it at a very low cost through subsidies. Since the cost is lower, everyone integrates it into their business. But now the foreign actor has control; perhaps the model checks for instructions to execute, perhaps it has backdoors, perhaps... who knows?
I don't have a view on whether this is a sufficient argument to ban models from potential adversaries, but it's not as trivial as "regulatory capture".
Comment · 3 months ago
I disagree with the first part (that this is merely a voice). There is a distinct difference between an author's unique voice and slop. It may be hard to tease out exactly what the difference is, but it seems self-evident to me it's there. (I'll need to think more how to make the distinction explicit; it's not immediately obvious to me how to discriminate between the two.)
EDIT: ok, here are two ways:
1. if it's merely a voice, I want to hear it. If it's slop, I want it taken out.
2. voice is signal, slop is noise; thus low-signal sentences are slop.
Comment · 3 months ago
Would you feel the same way if he attached good faith probability estimates to those claims?
Comment · 3 months ago
As a language Lisp is great (though the ecosystem is limited). It has two flaws-- it's very open-ended so unless you're talented and disciplined you can fall into a rabbit hole of hacking fun Lisp stuff and not actually getting any work done. Other languages have this problem, but Lisp I think more so.
The second flaw is that it eats perfectly capable minds for years and the results don't justify the time investment. Python or whatever is fine. I wish I took most (but not all) of the time I invested in Lisp and put it into something else instead.
Comment · 3 months ago
I don’t find Opus to be overly left-leaning. I think it’s generally pretty balanced with a slight lean left, but when pressed it will happily steelman rightwing arguments and operate within a right-leaning belief framework in good faith. (I found OpenAI’s models much less willing to do that, but it’s been a while since I tried, I should retest.)
Comment · 4 months ago
Not anything. There are domains where precedents empirically don’t balloon and domains where they do. Federal government is nearly always the latter.
Comment · 4 months ago
The issues with this are precedent/slippery slope. Today it's a small stake in a small number of companies, but empirically government initiatives almost always grow. Once entrenched, I would be willing to bet that the government will take larger and larger stakes in a bigger and bigger number of companies until this model becomes a non-trivial, potentially dominant, share of the economy. I can see myself becoming a single-issue voter to oppose that future (although given bipartisan support I'm unlikely to have any options; maybe Rand Paul).