Comment

Comment by carbocation

I would argue that this test isn't particularly informative. Given 5 attempts and 5 successes, even though the point estimate of accuracy is 1, the 95% CI ranges from 0.48 to 1:

    > binom.test(5,5,0.5)

     Exact binomial test

    data:  5 and 5
    number of successes = 5, number of trials = 5, p-value = 0.0625
    alternative hypothesis: true probability of success is not equal to 0.5
    95 percent confidence interval:
     0.4781762 1.0000000
In other words, we don't have enough data in that small sample to reject the possibility that the model is 50% accurate, much less 99.9% accurate.
parent carbocation 3 years ago

Discussion

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton