Because there are good and bad people in the world, and if I ask Claude "how do I make cocaine" and it gives me a thorough recipe, it doesn't know if I'm being intellectually curious or trying to become the next Pablo Escobar. They're Anthropic, and because they made Claude, they get to say what we can do with it. Which, I agree, is total bullshit. If I buy a Ford car and I can do whatever I want with it. Drive to work, drive for Uber, set it on fire; plowing into a crowd of protestors is covered by existing laws against vehicular manslaughter. Ford doesn't get a say. But because cars are so ubiquitous and a known technology, there's no blowback to Ford if I commit crimes with my car that I bought from them. If I commit crimes and Claude helped, Anthropic is gonna get sued (there's already litigation against the labs for deaths), and they don't want that.
In a properly free market, we'd have regulations that prohibit labs from vertical integration so they have to release and sell a model and another company hosts it and sell access to it. Americanism, America's implementation of capitalism doesn't have such regulations though (and how would that even work?) so we're stuck with Anthropic having that power.
The fear, justified or not, is that a terrorist is going to use Claude to make sarin nerve gas and bomb subway stations, or the digital equivalent. Could you live with that on your conscience? I don't think I could, but fortunately for me, I don't work at Anthropic so don't have to answer that real life trolly problem.
In a properly free market, we'd have regulations that prohibit labs from vertical integration so they have to release and sell a model and another company hosts it and sell access to it. Americanism, America's implementation of capitalism doesn't have such regulations though (and how would that even work?) so we're stuck with Anthropic having that power.
The fear, justified or not, is that a terrorist is going to use Claude to make sarin nerve gas and bomb subway stations, or the digital equivalent. Could you live with that on your conscience? I don't think I could, but fortunately for me, I don't work at Anthropic so don't have to answer that real life trolly problem.