I think we have to assume that the paraphrasing is at best a poor paraphrase and at worst completely misleading.
The paraphrased statement contains soft, non-deterministic hedges like "usually" and "sometimes", which would of course introduce uncertainty. Uncertainty -> low confidence.
The article strips out the actual prompt that went to Luna so we can't be sure they haven't ballsed the prompt just as they did with the paraphrase.
Compare the paraphrase to the language used in the actual paper they're trying to base their benchmark on:
> Bob is round.
> Alan is blue, rough and young.
> If someone is round then they are big.
> All rough people are green.
> Big people are not green.
This is obviously much easier to make certain statements about (the only thing we have to assume is that Bob and Alan are both "people").
The paraphrased statement contains soft, non-deterministic hedges like "usually" and "sometimes", which would of course introduce uncertainty. Uncertainty -> low confidence.
The article strips out the actual prompt that went to Luna so we can't be sure they haven't ballsed the prompt just as they did with the paraphrase.
Compare the paraphrase to the language used in the actual paper they're trying to base their benchmark on:
> Bob is round.
> Alan is blue, rough and young.
> If someone is round then they are big.
> All rough people are green.
> Big people are not green.
This is obviously much easier to make certain statements about (the only thing we have to assume is that Bob and Alan are both "people").