The primary thing that determines overall group performance (assuming no interpersonal biases like peer pressure or coercion, etc) would be the probability that any individual chooses the "right" answer. If this is greater than 0.5, then according to the Condorcet Jury Theorem, the limit of the group as a whole choosing the right answer is 1 as the group size increases.
Perhaps having a smaller size allows the "noise" in the group to still produce the "right" answer more often than it would in a larger group?
Perhaps having a smaller size allows the "noise" in the group to still produce the "right" answer more often than it would in a larger group?