Comment by __alexs

The whole idea of a confidence score as some abstract thing separate from statistics is wild to me.

Asking random experts their confidence is already pretty bad most of the time.

If you want a confidence score you need a way to consistently model the system you are trying to estimate the outcome of.

Anything else is just numerology.