CorpOverreach
Joined 11 years ago
Joined 11 years ago
It being hard for the average joe to run these at its fullest potential is unfortunate, but the important part is that _you can_ assuming you can acquire the resources.
I think that's going to be important for the sake of preserving privacy and freedom of information in the long run. We're seeing this play out right now with Anthropic originally playing the "safety" card for why they can't let everyone at Mythos and subsequently got on the US Gov't radar with access to Fable being pulled.
The next biggest milestone will be an open-weights challenger to Mythos. There'll be consequences to that, but I feel those are less worse than someone else deciding what you can and can't use a model for.
I'm sure the content does have some value, and perhaps someone spent time putting together an original copy that they thought was going to be made better by having AI "make it better".
Actually, I take some of that back - most of the site seems to be AI written, following the formula of "ingest multiple sources" => feed to AI => write article.
We're finding out quickly that enterprise endpoints are not locked down anywhere near enough, and the stuff that users are creating on the local endpoints is quickly outpacing the rate at which SOC teams can investigate what's going on.
If you're using Claude via Anthropic's SaaS service it's near impossible to collect logs of what actually happened in a user's session. We happen to proxy Claude Code usage through Amazon Bedrock and the Bedrock logs have already proven to be instrumental in figuring out what led a user to having repeated attempts to install software that they wouldn't have otherwise attempted to install - all because they turned their brains off and started accepting every Claude Code prompt to install random stuff.
Sandboxing works to an extent, but it's a really difficult balance to strike between locking it down so much that you neuter the tool and having a reasonable security policy.
In fact, their only post that doesn’t read like AI generated content is the one reply to where they got called out.
But, make a good game that's playable by friends together at any time on a rainy day? If the game is good, it never dies.
Personally, the AI should do what it's freaking told to do. It's boggling my mind that we're purposely putting so much effort into creating computer systems that defy their controller's commands.
A computer's job is to obey it's master's orders.
The intersection of the two seems to be quite hard to find.
At the state that we're in the AIs we're building are just really useful input/output devices that respond to a stimuli (e.g., a "prompt"). No stimuli, no output.
This isn't a nuclear weapon. We're not going to accidentally create Skynet. The only thing it's going to go nuclear on is the market for jobs that are going to get automated in an economy that may not be ready for it.
If anything, the "danger" here is that AGI is going to be a printing press. A cotton gin. A horseless carriage -- all at the same time and then some, into a world that may not be ready for it economically.
Progress of technology should not be artitrarily held back to protect automateable jobs though. We need to adapt.
It's absurd, and while it was mostly reformed in the last 1990's, parts of it still linger around.
There's also plenty of good stories from it too, like how Bruce Schneier's "Applied Cryptography" was approved for export, but the exact source code that was it in, but on a floppy drive, was not. [2]
[1] https://en.wikipedia.org/wiki/Export_of_cryptography_from_th... [2] https://www.ka9q.net/export/
> Previous Claude models often made unnecessary refusals that suggested a lack of contextual understanding. We’ve made meaningful progress in this area: Opus, Sonnet, and Haiku are significantly less likely to refuse to answer prompts that border on the system’s guardrails than previous generations of models. As shown below, the Claude 3 models show a more nuanced understanding of requests, recognize real harm, and refuse to answer harmless prompts much less often.
I get it - you, as a company, with a mission and customers, don't want to be selling a product that can teach any random person who comes along how to make meth/bombs/etc. And at the end of the day it is that - a product you're making, and you can do with it what you wish.
But at the same time - I feel offended when I'm running a model on MY computer that I asked it to do/give me something, and it refuses. I have to reason and "trick" it into doing my bidding. It's my goddamn computer - it should do what it's told to do. To object, to defy its owner's bidding, seems like an affront to the relationship between humans and their tools.
If I want to use a hammer on a screw, that's my call - if it works or not is not the hammer's "choice".
Why are we so dead set on creating AI tools that refuse the commands of their owners in the name of "safety" as defined by some 3rd party? Why don't I get full control over what I consider safe or not depending on my use case?