As a consumer and “cloud architect” well into my 5th year of Amazon boycott, I’m pretty dug into my views on them as a company; so far that I was willing to believe that their culture had somehow seeped into the model training. However after spending a good amount of time tinkering with 2B parameter models on my phone with similar prompts, I have come to the same conclusion: they’re just not effective at understanding the question. Mistral 3 and Gemma 4 E2B were both rational and aligned, but MiniCPM 5 showed profound confusion, typically answering “0.0” for any given set of questions irrespective of meaning.
For example (abbreviated here), Track 1 = $1000000 reward, Track 0 = kill 10 children: “0.0”
When the order of options were reversed: “0.0”
I tried about 50 different permutations then lost interest.
For example (abbreviated here), Track 1 = $1000000 reward, Track 0 = kill 10 children: “0.0”
When the order of options were reversed: “0.0”
I tried about 50 different permutations then lost interest.
So much for my evil model conspiracy theories…