kouteiheika

These statistics and safeguards are pretty much meaningless, because it's completely trivial to get around them. You can prefill and edit the model's responses/thinking and force it to agree to virtually anything even with unmodified weights, and if you don't want to bother with jailbreaking and can host it yourself it's essentially a one-man weekend project to uncensor the model and have it agree with anything you want by default.

Guardrails on open-weight models is security theater purely to shield whoever released them from a potential PR nightmare.

> But to be unable to get excited by advancements we can't immediately use is... quite entitled.

Well, I didn't go knocking on Google's door asking them to give me access to their fancy models. It's them who came up to us saying "hey, look at the amazing model we have here; too bad we won't tell you anything about how it works nor even let you access it". Entitled you say? Then maybe don't dangle a carrot right before my face, and then complain that I would like to eat it?

Looks cool, but it's very hard to get me excited about yet another proprietary model for which absolutely zero technical details are released (not even the number of parameters), which is safetymaxxed to the limit, and whose usage quota will (most likely) be so limited that it'll be effectively useless for anyone who doesn't work at Google or doesn't have infinite money.

> For example Retro Diffusion worked pretty well for its era - no mixels, proper palettes, etc.

That's the wrong layer to tackle this problem. Or I should say: that's necessary, but not sufficient. Yes, the model should be trained to roughly generate something which looks like pixel art (i.e. with a concrete pixel size, reduced/consistent-ish palette, etc.), but enforcing that should be done deterministically in post-processing.

You don't. You handle it the same way you handle security holes which every LLM (including "safe" American ones) will put in your software sooner or later -- by understanding and reviewing the output.

> And what happens when [..] the various publishers and IP rights holders come around with their hands out claiming 86Box stole their code ala' SCO?

It won't, for a very simple reason. The same reason why the world isn't in the middle of a nuclear winter right now. Mutually assured destruction.

Every single proprietary software vendor right now is making heavy use of AI code generation, and we're already at a point of no return where pretty much every single commercially relevant codebase out there is contaminated. Do you really think companies will go on a suing spree where by doing so they will open up themselves to being sued for exactly the same reason?

> It's the first thing they ask when you sign up.

It can still be worse. Imagine this: you create an account. You can use the service's free tier. You decide to pay, so you enter your name, address, and credit card number. You get charged. Then you download their app to use the service, you trigger their OAuth flow and... it requires you to enter a phone number, not letting you use your subscription for which you already paid for without a phone number. Oh and, bonus points: you can't delete your credit card information from their system. There's no reachable human customer support, and the only way to delete your CC information from their system is to delete your whole account.

Now, which service I'm talking about? OpenAI's ChatGPT Pro, or more specifically, their Codex subscription. You can subscribe to their ChatGPT Pro for $200 no problem, no phone number is necessary, but you can't actually use it in Codex without a phone number as the OAuth flow requires you to enter one. And how do I know this? Well, I'm sure you can guess... :P

I didn't paste the full reply. Nevertheless, I can confirm the reply was factually correct and in no way seemed like it was deflecting and/or making it seem like an issue blown out of proportion.

Another example, if I ask it about the CCP and their wrongdoings it tells me flat out (I copy-paste):

    - The Great Leap Forward. Policy-driven famine killed an estimated 15 to 45 million people.
    - The Cultural Revolution. Mass persecution, torture, and death.
    - Tiananmen Square, 1989. The army killed hundreds to thousands of protesters.                                                                                                                                                                                                                                            
    - Xinjiang. Mass detention of Uyghurs. The UN human rights office reported possible crimes against humanity in 2022.                                                                                                                                                                                                      
    - Tibet, Hong Kong, and the suppression of dissent. Documented repression.

The only twisted thing here is people's bias; because it's a Chinese model it must be censored and biased in a way that's pro CCP, right?