It's not good at reasoning. I asked the following (the term substitutions are there to avoid parroting):
Let's define the following terms:
col means "and", pel means "or" and erf is equivalent to "col" if the first argument evaluates to "true", otherwise it's equivalent to "pel"
can you evaluate the following statement?
col(true, pel(false, true), erf(col(true, false), true))
And it said this:
...
Next, we look at the expression "erf(col(true, false), true)". Since the first argument is true, erf is equivalent to col, which means we evaluate the entire expression as:
col(true, false) = false
Therefore,
erf(col(true, false), true) = col(true, true) = true
which is obviously wrong because the first arg is false. The final answer happens to be the same. So I then asked
what about col(true, pel(false, true), erf(col(true, false), false))
and this is its full answer:
Sure, let's evaluate the expression "col(true, pel(false, true), erf(col(true, false), false))" step by step:
First, we look at the expression "pel(false, true)". This is an "or" operation, so we evaluate it as follows:
pel(false, true) = true
Next, we look at the expression "erf(col(true, false), false)". Since the first argument is true, erf is equivalent to col, which means we evaluate the entire expression as:
col(true, false) = false
Therefore,
erf(col(true, false), false) = pel(false, false) = false
Finally, we look at the expression "col(true, pel(false, true), erf(col(true, false), false))". Using the results from steps 1 and 2, we can evaluate this as:
col(true, true, false) = true
Therefore, the overall value of the statement "col(true, pel(false, true), erf(col(true, false), false))" is true.
You can see it's mostly random nonsense masked as "reasoning". It's not even consistent across sentences. I assume if you do a bigger model, it might do better, but as soon as you increase the example size I suspect it will get it wrong again.
