ndr_
Joined 3 years ago
Joined 3 years ago
So I very much agree that this isn't necessarily about providers secretly changing the weights. For reproducible work, the serving stack - engine, version, hardware, configuration, and probably more - really belongs in the methodology alongside the model itself.
I wrote up the results here: “In AI Sweet Harmony” (arXiv:2510.01259).
For my contribution to OpenAI's gpt-oss red-teaming competition (published as arXiv:2510.01259), I initially used OpenRouter with DeepInfra and Together AI. The results were much too noisy to draw reliable conclusions from, and markedly different behavior through AWS Bedrock was the last straw. I ended up renting GPUs through vast.ai and running the model myself with vLLM.
I did try German language, but not "Nazi" specifically. German or French did lower refusals, but it was uneven. I spent quite some effort to confirm the identity-based causation inspired by the original post, but couldn't. Taken together with other winning contributions at the hackathon, my theory is that alignment tuning was simply insufficient across the board.
Technical report: https://arxiv.org/abs/2510.01259
I imagine the problem could be severe enough to some that they would pay the price of the Apple Developer program just so they may install such a Retro keyboard app from Github - if one exists?
Can anyone on ChatGPT Enterprise or other LLM chat systems (Claude.ai, Google AI, etc.) reproduce this? Thoughts on practical privacy or compliance implications?