Comment Comment by EagnaIonat Nice write up!One test I do is to give a common riddle but word it slightly to see if it can actually reason.For example:"Bobs dad has five daughters, Lala, Lele, Lili, Lolo and ???"The 20B model kept picking the answer of the original riddle, even after explaining extra information to it.The original riddle is:"Janes dad has five daughters, Lala, Lele, Lili, Lolo and ???" parent EagnaIonat 1 year ago
One test I do is to give a common riddle but word it slightly to see if it can actually reason.
For example:
"Bobs dad has five daughters, Lala, Lele, Lili, Lolo and ???"
The 20B model kept picking the answer of the original riddle, even after explaining extra information to it.
The original riddle is:
"Janes dad has five daughters, Lala, Lele, Lili, Lolo and ???"